The most successful product teams using Rasa apply software engineering best practices todeveloping their assistants, including:
- Jenkins Git Ssh Key
- Jenkins Generate Ssh Key For Git Free
- Generate Ssh Key Putty
- Jenkins Generate Ssh Key For Github
- Versioning training data and action code in Git
- Reviewing changes before they go into production
- Running automated tests on proposed changes
Rasa X encourages best practices by integrating itself into your existingdevelopment workflows, letting you:
- Automate data synchronization with your Git repository
- Automatically stay up to date with the latest state of the remote repository onyour Git server
- Annotate new data and push those changes to Git
Set yourself up for success by connecting your Rasa X server with Git to monitor,version, and test your training data using GitHub, GitLab, Bitbucket, Travis CI,CircleCI, Jenkins, etc.
- Connect your Rasa X Server to a Git Repository
Configuring SSH authentication between GitHub and Jenkins. Mohit Goyal CI/CD February 27, 2017 February 16, 2019 3 Minutes. Generate SSH Key on Jenkins Server. Using GITSSH to set credentials SSH Private key for Daniel’s Github. Jan 26, 2018 After the message 'Downloaded Successfully.' Check the box 'Restart Jenkins.' Set 'GITSSH' in Environment Variables. Jenkins's git use 'OpenSSH', not 'plink'. GitHub: Generating a new SSH key and adding it to the ssh-agent and adding a new SSH key to your GitHub account. Bitbucket: setup a SSH key. Git Jenkins ssh-agent.
In order to connect Rasa X with your assistant’s Git repository, you will need three things:
- A Rasa X instance running in server mode (local mode does not support Integrated Version Control)
- A Git repository containing a project in the default Rasa Open Source projectlayout
Note
When you connect your remote Git repository to Rasa X it will overwrite the training data,which is currently stored in Rasa X. Please use a fresh Rasa X instance or exportyour training data if you want to keep the old training data.
For Rasa X to correctly visualize and modify your AI assistant’s data, your projectneeds to follow the default Rasa Open Source project layout created byrasa init:
If you have just installed Rasa Open Source for the first time, you can run
rasainit
in a new Git repository to achieve this structure. If you have anexisting assistant that you’reconnecting to Rasa X, make sure to add it to a Git repository andreorganize the project if it does not match the above layout.- To connect your Git repository click on the branch icon and clickConnect to a repository.
- Configure the repository connection:
- SSH URL: Rasa X will clone the repository using the given SSH URL. Cloning viaHTTP is currently not supported.
- target branch: The target branch is the branch that Rasa X will
- use to show the initial data
- branch off from when you make new changes
- return to after you discard or push changes
- By default users can choose if they want to push their changes directly to thetarget branch or to a new branch. If want to disable pushing changes directly tothe target branch, select Require users to add changes to a new branch.
- Add the provided public SSH key to your Git server. This allows Rasa X toauthenticate with the Git server using its private SSH key. Please see thedocumentation of your Git server how to do so. We have linked the instructionsfor some common providers inAdd the Public SSH Key to Your Git ServerNoteIf you prefer to provide your own keys, please seeConnecting a Repository via the API.
- Once you added the public SSH key to your Git server, hit the Verify Connectionbutton. Rasa X will now show that it is connected to your repository.
- What to do next: Check out Using Integrated Version Control to understandhow to use Integrated Version Control as part of your process for improving your assistant.
You have to add the public key of the generated key pair to your Git server. Pleasemake sure to only give the key access to one specific repository instead of giving itglobal access to all of your Git repositories. For instructions specific to your Gitplatform, see below.
GitHub¶
Add the generated public SSH key as a
Deploykey
to your GitHub repository.See theGitHub docsfor more information on how to do so.GitLab¶
Add the generated public SSH key as a
Deploykey
to your GitLab repository.See the GitLab docsfor more information on how to do so.Bitbucket¶
Add the generated public SSH key as an
Accesskey
to your Bitbucket repository.See theBitbucket docsfor more information on how to do so.If you want to use self-generated SSH keys or prefer to use the Rasa X API, you canalso the Rasa X HTTP API to connect Rasa X to your Git repository.
- To authenticate your Rasa X server with the remote repository, you needto set up an SSH key that Rasa X can use for authentication.Please create a new, single-use SSH key for this (see instructions below).Also, make sure to restrict the SSH keys to only apply to your assistant’s repository.To generate a new SSH key pair follow these steps:
- Open the Terminal
- Execute the following command (make sure to not overwrite your own SSH keys):
This provides you a private (git-deploy-key
) and apublic (git-deploy-key.pub
) key in your current directory.NotePlease note that Rasa X currently does not support password protected privatekeys. - Save your repository information and private key to a file
repository.json
, in the format shownbelow. If your Rasa X server does not use HTTPS, we highly recommend doing this directlyon your server to avoid compromising the key. As the contents of this file willuploaded via a curl request, the directory where it is stored does not matter, butit is recommended to store the file somewhere secure.NoteThe target branch is the branch that Rasa X will- use to show the initial data
- branch off from when you make new changes
- return to after you discard or push changes
If you want to disable adding changes directly to the target branch, pleasespecifyis_target_branch_protected':true/false
in therepository.json
file.For example, yourrepository.json
might look like:WarningWhen connecting the Rasa X instance to a git repository, any training data or configuration filesstored in Rasa X will be overwritten by those in the Git repository. If you were using Rasa X tomanage your assistant before setting up Integrated Version Control, be sure to download the data beforecontinuing, so that the data is not lost. You can push the downloaded data from your machine to your Gitrepo before or after connecting it to Rasa X. - To authenticate with the Rasa X server you can use one of two methods:
- API token authentication: Get your API token by going to the model screen inRasa X and copying the
api_token
token from theUploadModel
command.Similar to the upload command, you add it with theapi_token
query parametershown in the curl command below. - JWT token authentication: Use your JWT access token. You can get it from theauthentication endpoint and pass itwithin the
Authorization
header.
Once you have prepared your chosen form of authentication, create the repository by executing the followingHTTP request from the directory that contains yourrepository.json
:NoteIf your Rasa X server runs on HTTPS, make sure to usehttps://
in the--url
parameter. - API token authentication: Get your API token by going to the model screen inRasa X and copying the
Once you’re set up with Git and your assistant is automatically imported, you can Enable Workflows tobegin learning from real users. For more information about how data synchronization works, seeUsing Integrated Version Control.
I recently got tasked with setting up a new Jenkins box within my organization, and having it work with our GitLab hosted Git repositories. We had previously had some rudimentary interaction between these two sides, with webhooks from our GitLab repo calling our Jenkins instance, using the GitLab Webhook Plugin. This worked alright, based on the branch specification in the Jenkins jobs it would perform the build/test when changes were made in the repo.
The downside to this method is that there isn’t any feedback in GitLab. I’ve recently worked on a project for my masters where we had a GitHub repo working with CircleCI, and with every commit to GitHub our build/test would run and we would get nice graphical statuses in GitHub. Below shows the nice status check:
It would sure be nice if I could get similar integration between GitLab and Jenkins. When I looked around at GitLab’s documentation, it mentioned that it supported connections to Jenkins, but only in the Enterprise Edition, which isn’t coming to our organization any time soon…bummer. It also mentioned that the Community Edition does work with GitLab’s own CI runner, but we have a bunch of other things that only run in Jenkins (we still have clearcase projects around here), so we couldn’t just move to that.
Luckily, while I was looking around for the Jenkins GitLab Webhook Plugin that we had used previously, I stumbled across the Jenkins GitLab Plugin. At first I thought it might just be another plugin that works similar to the webhook plugin, but after digging a bit deeper into the README on its GitHub Page, I realized that this was performing a similar service to what I was looking for!
To start the process, I installed Jenkins 2.7.2 as a service on windows (from the .msi installer), with the default plugins (includes the Git Plugin). I then installed the GitLab Plugin (version 1.4.0). While I’m naming versions, we’re currently running GitLab CE 8.11.2. The machine has Git 2.9.2 installed on it.
I next created a SSH key pair on the machine, which put the keys in C:UsersMY_USER.ssh (id_rsa for the private key and id_rsa.pub for the public key). I added the public key as a deploy key to my GitLab project(s). I then added my private key as a Jenkins Credential.
Enter the credential manager
Go into system credentials
Add a new credential
We need a SSH private key credential. I’m not sure the username matters, but I set it to “git” as that is what the login for gitlab uses. I then pointed it to my private key on the machine and gave it a description.
Now the key shows up in the system.
![For For](/uploads/1/2/6/8/126896254/372001401.png)
Once I had set up the ssh private key, which allows Jenkins to connect to the git host over ssh (i.e. git clone git@server:user/repo.git), I needed to set up API Access for Jenkins to get metadata from GitLab.
In GitLab go to your user profile.
Select “Access Tokens”.
Now make a new token, I gave it a descriptive name and made it expire at the end of the decade (I haven’t investigated the pros/cons of expiration date length yet).
You are then presented with a screen showing the token. Copy it now, as it won’t be accessible again! You can always create a new one if you mess it up though. (FYI I have revoked the token shown above)
Back in Jenkins’s System credentials add a new one of the type GitLab API token. Paste in the API token shown in the last step.
Now the API link has been added.
Once the API Access has been set up, we can configure the connection between Jenkins and GitLab. This happens in the Mange Jenkins -> Configure System menu.
Give it a name, the host where GitLab’s API lives, and the credential for the token created in the last step.
Now that our connection between Jenkins and GitLab is set up, we need to create a job. I’ve only worked with freestyle jobs, so if you’re doing a pipeline you’ll be on your own from here on out.
You’ll see that the connection you just made is automatically selected.
In the source code management section, enter the ssh URL of your repository. Select the ssh key generated above. Then we need to do the unique items.
We need to specify the name as “origin”, which will be used by the other sections. For the Refspec we need to input:
I haven’t completely figured out how this works, but it seems to be correct for my use cases.
For the Branch Specifier we need
origin/${gitlabSourceBranch}
which will be filled in based on the web hook we’ll be setting up next.For the trigger we want to set it up for changes going in to GitLab.
Finally, in the post build step we need to “Publish build status to GitLab commit”. This enables the feedback and gives us pretty indicators in GitLab.
That’s everything from the Jenkins side. Now we need to set up the GitLab side to trigger the builds. We’ll do this with webhooks, and we’ll need one per Jenkins job.
From the GitLab project you want to build, select the Webhooks option from the settings menu on the right.
You need to enter the URL of the jenkins server. The path is “project/JOB_NAME”. Select Push events and Merge Request Events.
Now everything should be set! If you push a commit to the repository, you should see the Jenkins job start running. Once the job completes, you should see the status next to the commit in GitLab.
Here’s a few commits with their build result status.
Clicking on the status icon shows some info about the build. Clicking on that Build ID (#6253) will take you to the Jenkins page for that build.
This setup works great for individual commits that are pushed into the repo. However, it also works fine for handling merge requests (despite some of the wording on the Jenkins GitLab Plugin’s readme page). I was able to have merge requests from separate branches in the same repo as well as from forked repos trigger builds and show the resulting status.
Jenkins Git Ssh Key
Merge request while the build is in progress.
Jenkins Generate Ssh Key For Git Free
A merge request with the completed build. The View details link will take you to the same build status page as above, and on to the Jenkins page for that build.
All in all, I’ve been very happy with the use of the Jenkins GitLab plugin to support CI between Jenkins and GitLab. It was definitely an unexpected bonus after reading about how you need the EE of GitLab to hook to Jenkins, apparently that isn’t completely true.
Generate Ssh Key Putty
Edit 2018–01–19: Thanks to Chelsea Greger for pointing out that the issue I mentioned in the above paragraph can be overcome by setting the job up as a parameterized job, with a parameter for “gitLabSourceBranch” (default to “master” if desired). This allows the job to be manually run.
Jenkins Generate Ssh Key For Github
To make more ease to run the jenkins job manually you can use Git Parameter Plugin that will list all branch and assign to a parameter. Then use this same parameter in 'Branches to build' configuration.