It seems that there are three big themes to be picked up next in this space:
Cleaning up tasks from the Atlassian suite migration
Delete older machines. I expect jira, confluence, id and crowd to not be needed anymore. That also includes their databases and database storage. That would include as well removing ansible code, archiving ID repo, the lot
Update documentation related to how we do our ID now: OpenMRS ID
Improve monitoring for this area
Define if OpenLDAP is what we want in the future. The main concern here is cost. If we go with other solutions, we may need to define the lifecycle of those users, so we limit ourselves to a sensible number of accounts at a time.
- ITSMOLD-4324Getting issue details... STATUS : if openldap is choosen to be kept, we will need to upgrade it
- ITSMOLD-4231Getting issue details... STATUS : if openlap is chosen to stay, we need to get those certificates to automatically restart the container in a way that will actually pick the new certificate. We may go with a cron task, may be easier than the letsencrypt hook. Potentially the ldap upgrade will help here
Verify the future of login of https://atlas.openmrs.org, that used to use our old ID. This system may be considered for sunset as well
Bamboo stability
bamboo is likely due to an upgrade! It may be a huge one
We may look if we can change any configuration on predator to make builds more stable https://marketplace.atlassian.com/apps/1212736/predator-plugin-for-bamboo?tab=overview&hosting=server
- ITSMOLD-4322Getting issue details... STATUS : Bamboo server and agents seem to run out of disk every so often due to logs. We can look at logrotate carefully
- ITSMOLD-4316Getting issue details... STATUS : Bamboo backups may not be working as desired
Check monitoring for those instances. Do they need more resources? Check build waiting time, do you need more agents? If we are willing to pay AWS, we could have elastic agents (not sure if desired)
Any other build improvements for reliability needed
Monitoring love
Datadog seems to be notifying things non stop. Do some machines need some more memory/CPU/disk? Do we need to do some cleanup?
- ITSMOLD-4149Getting issue details... STATUS : having datadog monitoring for unhealthy containers could be rather useful. For unhealthy containers, we may want to automatically restart them or something
- ITSMOLD-4319Getting issue details... STATUS : do all machines that need backup are deploying it as expected? Do we have good monitoring for them?
- ITSMOLD-4228Getting issue details... STATUS Pingdom is a paid integration. If our datadog isn’t as noisy anymore, we could potentially replace it altogether with slack, unless we actually want any folks on call
Miscellaneous tasks that could be picked as well
- ITSMOLD-4317Getting issue details... STATUS : this is worth testing now the Jetstream got upgraded. It may not be relevant anymore and could potentially be reverted
- ITSMOLD-4144Getting issue details... STATUS : potentially etherpad isn’t used anymore. On that case, archive the card and change docs to reflect that
- ITSMOLD-4143Getting issue details... STATUS : potentially not used anymore. On that case, archive the card and change docs to reflect that OpenMRSBot IRC Bots / https://github.com/openmrs/openmrs-contrib-itsmresources/wiki/Service-Chat-bots
- ITSMOLD-4318Getting issue details... STATUS It’s always necessary
Upgrading ansible
- ITSMOLD-4075Getting issue details... STATUS : not every DNS entry is on terraform, which is fine. But as you find more of those, you can add them into IaC
Reach out to centralised log services (e.g. splunk, sumologic, datadog) and ask if they’d be willing to provide us with an open source licence (preparation for future - ITSMOLD-3930Getting issue details... STATUS )
Suggested
Upgrading terraform (i.e., our infra was built with 0.12.31, current version is 1.9.3) lest we hit a point where functionality starts breaking and we’re forced to upgrade quickly.
Making our SSO more robust (we’ve managed to get KeyCloak working, but I’m not sure we’re fully divested from the old OpenMRS ID code, don’t have clear/easy mechanisms for granting permissions, don’t have an easy ± (semi-)automated way to mark accounts as spam
Better supporting our dev and CI processes (e.g., make sure devs reliably have the environments they need, fix issues with permissions getting messed up via docker on CI, etc.)
And there’s a long tail of other nice-to-haves (e.g., fixing things that are occasionally breaking like LetsEncrypt upgrades), though some of those might get fixed with upgrades.