TeamCity on Google Cloud
Deploying TeamCity using Cloud Deployment Manager
TeamCity is a popular continuous integration platform that I have encountered at a few organization, ranging from cloud solutions using C#, Java, Rails, NodeJS, and Go.
TeamCity has a lot of cool features, such as supporting child job that can inherit properties and steps from a parent job, and this can be configured using pipelines-as-code with a Kotlin DSL.
TeamCity on Google Cloud
If you would like to try TeamCity out on Google Cloud, there’s some scripts you can use easily deploy it. This guide walks you through using this setup, and some additional techniques to document the environment, should you want to convert this to other IaC (Infrastructure-as-Code) tools.
About Deployment Manager
Deployment Manager is an infrastructure deployment service that automates the creation and management of Google Cloud resources. It is similar conceptually to Cloud Formation for AWS, or other open source tools like Terraform or Ansible for cloud provisioning in general.
This solution is rather complex and having some understanding of Python and Jinja would be useful. As one redditor posted:
“In general, it really stinks. It has very poor support for most google services and is kind of a pita to use” - red88888888 (Oct 14, 2021)
Recently, Google has been promoting Terraform and KRM (Kubernetes Resource Model) over Deployment Manager. Another redditor noted that:
“The focus has shifted to Config Connector, its managed version Config Controller, and Terraform.
Config Connector and Controller in particular are developed by the same team that’s supporting Deployment Manager” - MrTrustor (Oct 14, 2021)
The idea behind those products is that the best thing about Kubernetes is its API model (see Kubernetes Resource Model), and so they want to let you leverage that for all your cloud resources. This also lets you leverage the Kubernetes ecosystem for managing your cloud resources (things like kustomize or helm for example).
Tool Requirements
- Google Cloud SDK 394.0.0 (
gcloud): to use deployment manager and other cloud operations - GNU Sed 4.8 (
sed): standard on Linux, on macOS, use Homebrew:brew install gnu-sed && alias gsed=sed
The Scripts: Deploying TeamCity
These scripts have not been touched in two years and as such, they do not work. There are a bunch of issues, such as use of deprecated APIs, hard-coding path to fetch images that are inaccessible, and so on.
In in event, I worked the patches needed and updated the steps required, and made this available below.
⚠️ WARNING: These Deployment Manager scripts will place both the virtual machine instance and the MySQL database on the public internet, which makes them vulnerable to attack. This is definitely not best practices in deploying a service, and should never be used in production. You may want to modify the firewall rules to only allow traffic from your public IP address, e.g. https://www.whatismyip.com/.
⚠️ WARNING: The systemd unit scripts used to launch the TeamCity Server docker container will run this as root, so that it can access mounted directories. This violates best practices in security for running docker containers.
📔 NOTE: This will take some time to run these scripts, greater than 20 minutes.
Once completed you can log into the TeamCity server. First get the IP address:
gcloud compute instances list --project $PROJECT_IDYou will be greeted with a license agreement, and then prompted to create the administrative account, and then brought to the Getting started with TeamCity screen, where you can begin creating projects:
Logging into the server
If you need to log into the system, you can do so with:
gcloud compute ssh \
--zone us-central1-a \
--project my-cicd-project \
tc-server-vm \When logged in you can check on the services:
systemctl list-units --state=failed
sudo journalctl -u teamcity-server.serviceBonus: Exporting to Terraform
Now that this infrastructure was created, export it to Terraform scripts using the tool Terraformer.
Tool Requirements
- Google Cloud SDK 394.0.0 (
gcloud): to use deployment manager and other cloud operations - Terraform v1.2.6 (
terraform) and optionally tfenv 2.2.3 (tfenv). - Terraformer v0.8.21 (
terraformer)
Installing and Running Terraformer
Terraformer is not the easiest tool with which to get started. I created a script that will run through the steps on Linux (tested on Pop!_OS). This should work for macOS.
📔 NOTE: The steps below assume the architecture is amd64. If you are using, for example, arm64, change to that architecture if available.
📓 NOTE: This should work on macOS. You can use Homebrew to install some components. See the comments for alternative install methods.
These will give you hints on how create similar infrastructure without the need for Deployment Manager, which is no longer in favor at Google. When completed, you can check out the directory ./generated to look at the Terraform scripts:
generated/
└── my-cicd-project
├── addresses
│ └── us-central1
│ ├── compute_address.tf
│ ├── outputs.tf
│ ├── provider.tf
│ └── terraform.tfstate
├── cloudsql
│ └── us-central1
│ ├── outputs.tf
│ ├── provider.tf
│ ├── sql_database_instance.tf
│ ├── sql_database.tf
│ └── terraform.tfstate
...(snip)... ├── instances
│ └── us-central1
│ ├── compute_instance.tf
│ ├── outputs.tf
│ ├── provider.tf
│ └── terraform.tfstate
└── networks
└── us-central1
├── compute_network.tf
├── outputs.tf
├── provider.tf
└── terraform.tfstate
Resources
General
- A Brief History of DevOps Part III: Automated Testing and Continuous Integration (2018)
- Oracle hands Hudson to Eclipse, but Jenkins fork seems permanent (2011)
Market
Google Deployment Manager
Fatcar Linux (previously CoreOS Linux)
Conclusion
I wanted to write some articles on TeamCity as it is hard to find articles and support outside of Jetbrains content. Despite having a Freemium model to distribute TeamCity, popular open source, like Jenkins, have far more community support, plugins, and documentation.
TeamCity vs. Jenkins
TeamCity (2006) shares a history with Cruise Control (2001) and Hudson (2005), as they platforms that run as a Java servlet container on Tomcat. Jenkins (2011) split from Husdon after Oracle’s stewardship of Husdon related to the acquisition of Sun Microsystems.
Both TeamCity and Jenkins are similar, in that they are configured through a graphical web interface instead of configuration files or automaton and both platforms support a robust plugin system to extend the product.
Where TeamCity and Jenkins differ is that Jenkins is essentially useless without plugins, while TeamCity by itself comes with a lot of functionality found in numerous Jenkins plugins. Both platforms support Pipelines-As-Code: Jenkins with a DSL using Groovy and TeamCity with a DSL using Kotlin.
Jenkins, through plugins, has support to configure the server using configuration files (see jasc), while TeamCity unfortunately requires manual button-clicking for configuration of the server through a web interface.
TeamCity supports arranging jobs (called projects) in a hierarchy, where child sub-projects can inherit properties and steps from the parent project; Jenkins does not support this organization, so for this reason, the list of jobs can quickly become quite voluminous, saturated, and difficult to maintain.
Final Thoughts
I hope this is useful for others to get started with TeamCity and manage slightly more robust way than just running containers on a local laptop.
The teamcity-google-teamplate repository is really the only thing available from Jetbrains for Google Cloud. The repo has not been updated in a while, outside of removing the “JB|Official” badge from the README.md. The current state of the repo is that it DOES NOT WORK. So I added some edits (using sed) for some minimal fixes to get this functional again.
As Deployment Manager is, well, complex and kludgy, I added a section to help get off these scripts and export infrastructure to Terraform using the terraformer tool. Once I get a block of time, I would like to redo this with better security practices, like not exposing the database to the public internet, for instance.
Many of these things mentioned in the article have been communicated to https://twitter.com/teamcity and https://twitter.com/jetbrains, and there hasn’t been any response.
Thank you for following.
