HTTP Triggering Jenkins Parameterized Build

Preface

Business-driven, no drive no move

I. Background

Using Selenium for frontend automated testing, hoping to get a test report with every code commit:

1.push/merge remote code

2.automatically run test cases

3.email notification of test results

Webhook connects Git operations and the build process. This way, every time there's a Git operation like push/merge, Webhook will (request to) notify the build service, then execute the entire build process, and send email notification after build completion.

Originally Jenkins already provides a one-stop service, just needs simple configuration. Webhook triggers Jenkins job, synchronously builds and sends email.

But due to the particularity of the frontend environment (test cases need to run in browser environment), and test results are also in JS hands, there's the problem of how to notify Jenkins after asynchronously completing test cases.

Use Selenium to start a browser, visit the test page, wait for all cases to complete (including async cases), only then get test results. But the build script may finish before the browser even opens, ready to send email but test cases haven't finished running.

So first separate the email sending task as an independent job, specifically responsible for sending emails. Then only missing is JS notifying Jenkins to send email.

II. Job Specifically for Sending Emails

Configure SMTP

Fill in SMTP related configuration in "Jenkins/System Management/System Settings", and System Admin Email Address, for example:

Mail Notification
---
SMTP Server         smtp.163.com
Use SMTP Authentication
Username             xxx
Password              ***

Jenkins Location
---
System Admin Email Address  user @163.com

At the bottom of Mail Notification section there's Test configuration by sending test email option, fill in recipient and try sending, receiving it is good.

P.S. Especially note to fill in System Admin Email Address, otherwise emails will never be sent (no sender filled).

Then configure "Extended E-mail Notification", post-build email notification content is set here, for example:

Extended E-mail Notification
---
SMTP server             smtp.163.com
Default Content Type    HTML(text/html)
Use SMTP Authentication
User Name               xxx
Password                ***

Check "Enable Debug Mode" at bottom to facilitate troubleshooting email sending failures.

Parameterized Build

Set parameter name, default value and description in "General/This project is parameterized", for example:

String Parameter
---
Name       autoTestResult
Default Value     Auto test failed
Description       Auto test results, which test cases failed

Fill in token in "Build Triggers/Trigger builds remotely (e.g., from scripts)" as build token, for example:

Authentication Token  mail

Execute shell in "Build" (for testing parameter passing):

echo $autoTestResult
echo 'hoho, the end'

Add "Editable Email Notification" in "Post-build Actions", fill in recipient/content items, for example:

Project Recipient List    user @163.com
Default Subject           autoTest Notification
Default Content           $autoTestResult

Then add "Always" in "Advanced Settings/Triggers", by default only sends email when build fails, change to send unconditionally.

Then try manually triggering (browser access):

http://localhost:2017/job/mail/buildWithParameters?token=mail&cause=shoujian&autoTestResult=allpassed

If everything is normal, the "Console Output" panel of this build will show following log:

Started by remote host 0:0:0:0:0:0:0:1 with note: shoujian
Building in workspace /Users/Shared/Jenkins/Home/jobs/mail/workspace
Checking for pre-build
Executing pre-build step
Checking if email needs to be generated
No emails were triggered.
[workspace] $ /bin/sh -xe /Users/Shared/Jenkins/tmp/hudson5065606977113971836.sh
+ echo allpassed
allpassed
+ echo hoho
hoho
Checking for post-build
Performing post-build step
Checking if email needs to be generated
Email was triggered for: Always
Sending email for trigger: Always
messageContentType = text/html; charset=UTF-8
  Collecting change authors...
    build: 17
Adding recipients from project recipient list
Adding recipients from trigger recipient list
Successfully created MimeMessage
Sending email to: user @163.com
Finished: SUCCESS

Then you will receive an email:

Subject: autoTest Notification
Content: allpassed

P.S. If following the steps above one by one, everything should be normal. If error or no email received, please check Jenkins Email sending fails

III. HTTP Triggering Jenkins Email Job

JS sending HTTP request to trigger Jenkins task will encounter 3 problems:

• CSRF protection

• CORS

• Login verification (Basic Auth)

By default there's CSRF protection and cross-origin restrictions, and login required (but supports Basic Auth).

If going through server intermediary, these are not problems, get crumb to pass CSRF, unrestricted cross-origin, header verification. But considering build tools only run in test environment, no need to go around, just turn off CSRF protection, enable CORS whitelist, finally JS can use XHR to set header to complete Basic Auth login.

Turn Off CSRF Protection

In "Jenkins/Manage Jenkins/Configure Global Security" uncheck Prevent Cross-Site Request Forgery exploits.

This way no need to get crumb verification, test environment, risk is not significant.

Enable CORS Whitelist

There's a CORS plugin specifically for this: CORS support for Jenkins

Fill in cross-origin restriction related response headers in "Jenkins/System Management/System Settings/CORS Filter", for example:

Is Enabled
Access-Control-Allow-Origins    http://localhost:3000
Access-Control-Allow-Methods    POST
Access-Control-Allow-Headers    Authorization
Access-Control-Max-Age          3600

P.S. If multiple values, separate with ,, for example POST,GET.

Add POST http://localhost:3000 to whitelist, allow XHR to read Authorization field in response header, ID validity is 3600s (1 hour).

P.S. For more CORS information, please check Cross-Origin Resource Sharing CORS Detailed Explanation

XHR Login

new Image() is most convenient, but no way to do Basic Auth. XHR GET also works, but URL length limit is always annoying, so choose XHR POST, example as follows:

var xhr = new XMLHttpRequest();
xhr.onreadystatechange = function() {
    if (xhr.readyState === 4) {
        if((xhr.status >=200 && xhr.status < 300 ) || xhr.status == 304 ){
            console.info('jenkins trigger successful');
        } else {
            console.info('jenkins trigger failed ' + xhr.status + '\n' + xhr.responseText);
        }
    }
};

var result = 'allpassed';
var url = 'http://localhost:2017/job/mail/buildWithParameters?token=mail&cause=shoujian&autoTestResult=' + result;
xhr.open('POST', url, true);
// Basic Auth
var username = 'jenkins', password = '2017';
xhr.setRequestHeader('Authorization', 'Basic ' + btoa(username + ':' + password));
xhr.send(null);

After all test cases complete, POST test results to notify Jenkins to send email. At this point frontend automated testing can run.

P.S. For more XHR Basic Auth information, please check Jenkins json REST api with CORS request using jQuery

PHP Triggering Jenkins Parameterized Build

If considering security risks, or hoping to do more things after running test cases, can trigger build from server, PHP code as follows:

header('Access-Control-Allow-Origin:*');

// Trigger build
// exec("curl -X GET \"http://127.0.0.1:2017/job/mail/build?token=mail&cause=nocause\" --user user:passwd", $res, $rt);
// Parameterized build
exec("curl -X GET \"http://127.0.0.1:2017/job/mail/buildWithParameters?token=mail&cause=nocause&autoTestResult=allpassed\" --user user:passwd", $res, $rt);
if ($rt !== 0) {
    echo 'Error occurred<br>';
    var_dump($res);
}
else {
    echo 'ok';
}

exec calls curl to send GET request, POST method is similar, no need to elaborate.

P.S. For more information about remotely triggering builds, please check official documentation: Remote access API

IV. Final Words

Jenkins handles many detailed tasks, more convenient than manually implementing such a build service, but the more configurable, the lower the control. To complete some simple operations, have to resort to plugins or work around ways.

Anyway, automatically running test cases to ensure infrastructure stability is excellent, get it going.