Sales +1-855-261-3391 Chat Blog Partner Support Status  |  Download

Networking Unplugged

Perspectives on cloud networking and software-defined WANs

Posted August 14, 2014
Comments 0 Comments

Secure Cloud Bursting (Part 3): How to Deploy a Jenkins Slave to a Secure Cloud

A lot of companies pay for unused cloud build infrastructure, including us. As a big proponent of efficiency and agile development, I knew there was a way with some of the new technology flying around to only pay for the build capacity we use at any given time. I just had to figure out how. 

Using Pertino in conjunction with Jenkins and Docker, I engineered a way to securely cloud burst, maximizing cost efficiency and reducing infrastructure complexity. I am sharing this process with you over a four-part blog series. Part three of this series walks you through the third step of the secure cloud bursting process—How to deploy a Jenkins Slave to a secure cloud.

If you missed post one (how to build a Jenkins Master) or post two (how to build a Jenkins Slave and connect it back to the Master), you will want to go back and catch up before beginning the steps in this post. Don't miss the final blog of this four-part series—How to perform a Jenkins initiated cloudburst. 

Note: This whole process will cost you less than $10 depending on how many times you want to repeat it. If you do it perfectly the first time, you're looking at a bill of less than $1.

1. Create a Digital Ocean Account by following the instructions. 

2. Obtain your Digital Ocean credentials.

Note: You must use API v1.0.

a. Click Generate New Key.

b. Record the Client ID and API Key for later use.

3. Now, you need to install Pertino.

a. Navigate to the Pertino trial install page.

b. Sign up via the form.

c. Follow the setup wizard.

d. For network name, use “Pertinodemo”.

e. Download the Pertino app using the APT install instructions.

f. The password in step 1 is your superuser password.

g. The username in step 4 is the email you used during Pertino sign up.

h. The password in step 4 is the Pertino password you created during sign up.

i. Once Pertino is installed, return to the web app and click next.

j. Click Close.

k. Click the orange DEVICES ONLINE box.

l. Click on the device (should be the only one).

m. Record the hostname. Mine is: ubuntu.xmfwkm.pertino.net.

4. Next, you need to unload the Jenkins Master and Slave.

a. Unload Jenkins-Master.

sudo docker ps -q

The values returned are the container IDs for the Jenkins Master and Slave.

b. Do the following for both values:

sudo docker stop <container ID>

5. Now, we're going to build a burstable slave.

a. Download the newest files supplied by Pertino: 

- demo/Dockerfile

- demo/jenkins-slave/Docker

-demo/jenkins-slave/run_slave.sh

b. In the demo directory, run:

sudo docker build -t iteration .

c. Change to the Jenkins-slave directory.

cd jenkins-slave

d. Into the Jenkins-slave directory, copy/paste runslave.sh.

e. Edit run_slave.sh and replace <your host> with the hostname from the Pertino setup (step 3m).

f. Build the slave.

sudo docker build -t jenkins-slave .

6. And build a new Jenkins Master.

a. Download the lastest demo/jenkins-master/Docker file.

b. Change directory to jenkins-master.

cd demo/jenkins-master

c. Make directory Jenkins Slave.

mkdir AddJenkinsSlave

d. Download the first config.xml file into directory AddJenkinsSlave.

e. Change directory back to jenkins-master.

cd demo/jenkins-master

f. Make directory called DeleteJenkinsSlaves.

mkdir DeleteJenkinsSlaves

g. Download the second config.xml file into DeleteJenkinsSlaves.

h. Change directory back to jenkins-master.

cd demo/jenkins-master

i. In the Jenkins-master directory, run

sudo docker build -t jenkins-master .

7. Now, run the Jenkins Master.

sudo docker run -p 8080:8080 -p 6022:22 -p 54525:54525 -h ${HOSTNAME} -d jenkins-master

Successful Master Relaunch

8. Next, we need to preload the jobs into the Jenkins Master.

a. Change directory to jenkins-master.

cd demo/jenkins-master

b. From the directory /demo/jenkins-master, issue the following two commands:

Note, use the password from the master dockerfile: 12345six.

rsync -Rave "ssh -p 6022" AddJenkinsSlave/config.xml jenkins@localhost:/var/lib/jenkins/.jenkins/jobs

rsync -Rave "ssh -p 6022" DeleteJenkinsSlaves/config.xml jenkins@localhost:/var/lib/jenkins/.jenkins/jobs

9. Next, we need to install the Jenkins plugins—Groovy and Groovy Label Assignment:

a. Navigate to the Jenkins webpage (http://localhost:8080).

b. Click Manage Jenkins.

c. Navigate to Manage Plugins.

d. Navigate to the Available tab.

e. Search for Groovy in the filter.

f. Select “Groovy plugin”.

g. Select “Groovy Label Assignment plugin”.

h. Click Download now and install after restart.

i. Verify successful plugin download.

Groovy Plugin Download Verification

j. Click Restart Jenkins when installation is complete and no jobs are running.

k. Allow Jenkins to shut down.

10. Build and run the Docker registry.

a. From the command line, obtain Docker container for the "Registry".

sudo docker pull registry:latest 

Build & Run Docker Registry

b. Run the registry on the Pertino interface.

IPADDRESS=`ifconfig pertino0 | grep "inet addr" | awk -F: '{print $2}' | awk '{print $1}'`

sudo docker run -p ${IPADDRESS}:5000:5000 -h ${HOSTNAME} -d registry

c. Push your Jenkins Slave image into the registry by replacing <FQDN> below with the hostname obtained during the Pertino setup (Step 3m).

sudo docker tag $(sudo docker images | grep "^jenkins-slave" | awk 'BEGIN { FS = " *" } ; {print $3}') <FQDN>:5000/jenkins-slave

sudo docker push <FQDN>:5000/jenkins-slave

Note: You can continue to step 11 while step 10c is completing.

11. Input your specific credentials into the jobs.

a. Navigate to the Jenkins webpage (http://localhost:8080).

b. Click AddJenkinsSlave.

c. Click Configure.

d. Scroll down to Execute shell.

e. Replace <yourClientId> and <yourApiKey> with the Digital Ocean information recorded in step 2.

Digital Ocean API Credentials

f. Add your Pertino credentials. Replace <yourPertinoEmailHere> with your email and <yourPertinoPasswordHere> with your password.

Pertino Credentials

g. In the statement “docker pull <FQDN>:5000/jenkins-slave”, replace <FQDN> with the hostname value from your Pertino account setup (Step 3m).

h. In the statement "sudo docker run --privileged -p \${IPADDRESS}:9022:22 -h \${HOSTNAME} -d <FQDN>:5000/jenkins-slave", replace <FDQN> with the hostname value from your Pertino account setup (Step 3m).

FQDN Credentials

i. Click Save.

j. Navigate back to the Jenkins webpage (http://localhost:8080).

k. Click DeleteJenkinsSlave.

l. Click Configure.

m. Scroll down to Execute shell.

n. Replace <yourClientId> and <yourApiKey> with the Digital Ocean information recorded in step 2.

o. Click Save.

12. Now, standardize the JNLP port.

a. Navigate to the Jenkins webpage (http://localhost:8080).

b. Click Manage Jenkins.

C. Click Configure Global Security.

d. Select Enable Security.

e. For TCP port JNLP slave, select fixed and enter "54525".

f. Click Save.

13. You can now run your add slave job.

a. Navigate to the Jenkins webpage (http://localhost:8080).

b. Click AddJenkinsSlave.

c. Click Build with parameters.

d. Enter “-MyTest-1” for uniquifyhost.

e. Click Build.

f. Navigate back to the Jenkins webpage (http://localhost:8080).

Notice the build status in the build executor status dashboard.

Build Executor Status

g. Wait for the job to complete. (This will highly depend on your connection speed.)

Note: You can watch the status of the build output.

i. Navigate to the Jenkins webpage (http://localhost:8080).

ii. Click AddJenkinsSlave.

iii. Click the top item in the build history.

iv. Click Console Output.

h. You should now have a slave named slave-auto-MyTest-1. (Mine is named slave-auto-MyTest-2-1 because I ran this more than once.)

Slave-Auto-MyTest-1

Note: This slave is also now in Digital Ocean.

New Slave in Digital Ocean

And Pertino. New Slave in Pertino

14. Now you can delete the Jenkins Slave you just created.

a. Navigate to the Jenkins webpage (http://localhost:8080).

b. Click DeleteJenkinsSlave.

c. Click Build with Parameters.

d. Enter "MyTest".

e. Click Build.

f. Navigate back to the Jenkins webpage (http://localhost:8080).

g. You can see the build status in the build executor dashboard.

h. Once the job completes, the machine should now be removed from Digital Ocean and offline at Pertino.

If you've been following along, you've now built a Jenkins Master and a Jenkins Slave, attached the Jenkins Slave to the Master and deployed the Jenkins Slave to a secure cloud. Technically, by testing the Jenkins Slave deployment in blog three, you performed a single cloudburst. The only step remaining in our secure cloud bursting process is performing a Jenkins initiated multi-cloudburst, which we will cover in our final blog of the series.

Luke Woydziak

Luke Woydziak

Networking Team Lead

Pertino is my seventh tech company, and I've loved every minute of it. After a career as a professional Track and Field Athlete, I've taken my passion and energy and directed it toward building and leading a team that solves complex engineering problems.

comments powered by Disqus
Chat now Click to Chat