Register - Outages and performance degradations

Service	Date/time UTC	Resolution	Cause	Comments
Confluence	21 Sep 2016			https://talk.openmrs.org/t/openmrs-wiki-down/8062
Sonar	22 Sep 2016 ~11am	Restart service		Sonar is not on status page Builds failed due to outage
Jira	22 Sep 2016 22:44	restart service	Out of memory	Click here to expand... 2016-09-22 21:10:15,849 ajp-bio-8041-exec-560 WARN anonymous 1266x78860x6 - 50.23.94.74 /secure/Dashboard.jspa [renderer.internal.local.LocalGadgetSpecFactory$Parser] Could not retrieve gadget spec https://issues.openmrs.org/rest/gadgets/1.0/g/com.atlassian.jira.gadgets:filter-results-gadget/gadgets/filter-results-gadget.xml from plugin-provided gadget spec store: java.lang.OutOfMemoryError: Java heap space 2016-09-22 21:10:15,925 ajp-bio-8041-exec-547 WARN anonymous 1266x78862x8 - 188.138.124.110 /secure/Dashboard.jspa [renderer.internal.local.LocalGadgetSpecFactory$Parser] Could not retrieve gadget spec https://issues.openmrs.org/rest/gadgets/1.0/g/com.atlassian.jira.gadgets:filter-results-gadget/gadgets/filter-results-gadget.xml from plugin-provided gadget spec store: java.lang.OutOfMemoryError: Java heap space Exception in thread "ajp-bio-8041-Acceptor-0" java.lang.OutOfMemoryError: Java heap space Exception in thread "ajp-bio-8041-exec-433" java.lang.OutOfMemoryError: Java heap space
Confluence	22 Sep 2016 23:26	Restart Service	Added new profiling flag
Jira	23 Sep 2016 23:38	Restart Service	Crashed
Bamboo	27 Sep 2016	Kill old service and start again	Crashed?	For some reason, Bamboo server was printing some very weird error messages to the logs: Also, service stop didn't work, I had to use kill -9. I forgot to reenable the agents, bringing the outage to pretty much a day.