|
| 1 | += One way Jira sync tool |
| 2 | + |
| 3 | +The sync tool uses web hooks and Jira REST API to perform the one way sync of upstream Jira to the downstream one. |
| 4 | +User actions in the upstream Jira trigger web hooks, which send the events to the sync tool. |
| 5 | +Sync tool uses the REST API to get the required information from the upstream Jira and |
| 6 | +sends it over to the downstream one. |
| 7 | + |
| 8 | +image::assets/images/sync-tool.png[] |
| 9 | + |
| 10 | +== Configuring web hooks on the upstream side |
| 11 | + |
| 12 | +In the upstream Jira (the source/original) configure a webhook per project you want to be synced. |
| 13 | +The tool listens for a web hook events at `/api/jira/webhooks/{PROJECT-KEY}`. |
| 14 | + |
| 15 | +Set up the secret, while creating the web hook so that the requests could be signed and checked, see https://developer.atlassian.com/cloud/jira/platform/webhooks/#secure-admin-webhooks. |
| 16 | +If secret was configured on the web hook side, signature verification can be enabled per project through: |
| 17 | + |
| 18 | +[source,properties] |
| 19 | +---- |
| 20 | +jira.project-group."project-group-name".projects.PROJECT-KEY.security.enabled=true |
| 21 | +jira.project-group."project-group-name".projects.PROJECT-KEY.security.secret=secret-you-have-specified-in-the-web-hook |
| 22 | +---- |
| 23 | + |
| 24 | +See also: |
| 25 | + |
| 26 | +* https://developer.atlassian.com/server/jira/platform/webhooks/ |
| 27 | +* https://developer.atlassian.com/cloud/jira/platform/webhooks/ |
| 28 | + |
| 29 | +== User configuration |
| 30 | + |
| 31 | +This tool requires two accounts provided per project group. One account is to access the upstream Jira while the other one |
| 32 | +is for the downstream Jira. |
| 33 | + |
| 34 | +Users are specified in the project group configuration in the `application.properties` file: |
| 35 | + |
| 36 | +[source,properties] |
| 37 | +---- |
| 38 | +jira.project-group."project-group-name".source.api-uri=https://your-upstream-jira-instance/rest/api/2 |
| 39 | +# if the Jira instance works with Basic authentication then username/PAT should be supplied: |
| 40 | +jira.project-group."project-group-name".source.api-user.email=your-user-name-of-service-a-user |
| 41 | +jira.project-group."project-group-name".source.api-user.token=your-personal-auth-token-for-the-above-user |
| 42 | +jira.project-group."project-group-name".destination.api-uri=https://your-downstream-jira-instance/rest/api/2 |
| 43 | +# if the Jira instance works with bearer tokens only, and basic auth is disabled, |
| 44 | +# specify the login kind and provide PAT: |
| 45 | +jira.project-group."project-group-name".destination.login-kind=BEARER_TOKEN |
| 46 | +jira.project-group."project-group-name".destination.api-user.token=$your-personal-auth-token-for-the-service-user |
| 47 | +---- |
| 48 | + |
| 49 | +Please see the `JiraConfig` javadocs for more details on the configuration properties. |
| 50 | + |
| 51 | +See also: |
| 52 | + |
| 53 | +* https://confluence.atlassian.com/enterprise/using-personal-access-tokens-1026032365.html |
| 54 | + |
| 55 | +== Further project group configuration |
| 56 | + |
| 57 | +With users configured the next step would be to define the projects and how they are mapped. |
| 58 | + |
| 59 | +[source,properties] |
| 60 | +---- |
| 61 | +jira.project-group."your-project-group".projects.YOUR-SOURCE-PROJECT-KEY.project-id=downstream-project-id |
| 62 | +jira.project-group."your-project-group".projects.YOUR-SOURCE-PROJECT-KEY.project-key=DOWNSTREAM-PROJECT-KEY |
| 63 | +jira.project-group."your-project-group".projects.YOUR-SOURCE-PROJECT-KEY.original-project-key=UPSTREAM-PROJECT-KEY |
| 64 | +jira.project-group."your-project-group".projects.YOUR-SOURCE-PROJECT-KEY.security.enabled=true |
| 65 | +jira.project-group."your-project-group".projects.YOUR-SOURCE-PROJECT-KEY.security.secret=jira-webhook-secret |
| 66 | +---- |
| 67 | + |
| 68 | +=== Mapping |
| 69 | + |
| 70 | +The tool takes the simple approach to mapping users, issue types, etc between Jira instances. |
| 71 | +Since all of these can differ between Jira instances, and then on top of that would be limited by the project scheme |
| 72 | +applied to a particular project, it is simpler for the tool just to ask the user to provide the direct mapping |
| 73 | +instead of guessing it. This section is here to provide some tips on how to get the info you need. |
| 74 | + |
| 75 | +==== Users |
| 76 | +Use the `jira/people/search` or `rest/api/2/user/search?username=...` to find the users. |
| 77 | + |
| 78 | +[source,properties] |
| 79 | +---- |
| 80 | +jira.project-group."your-project-group".users.mapping."source-user-id"=destination-user-id |
| 81 | +---- |
| 82 | + |
| 83 | +Depending on the jira instance type you may also need to change |
| 84 | +the property name that is used to specify the user in the downstream request. |
| 85 | +By default, the tool will try to use the `accountId` property, but for some Jira instances you may need to send |
| 86 | +the user id/name/smth-else through a different property, e.g. `name`. To do that add the following property: |
| 87 | + |
| 88 | +[source,properties] |
| 89 | +---- |
| 90 | +jira.project-group."your-project-group".users.mapped-property-name=name |
| 91 | +---- |
| 92 | + |
| 93 | +==== Priorities |
| 94 | +Review the project settings summary (e.g. `https://your-jira-instance/plugins/servlet/project-config/your-project-key/summary`). |
| 95 | +Use `rest/api/2/priority` REST API to get the list of available priorities and pick the ones defined in the project scheme. |
| 96 | + |
| 97 | +[source,properties] |
| 98 | +---- |
| 99 | +jira.project-group."your-project-group".priorities.mapping."source-id"=destination-id |
| 100 | +---- |
| 101 | + |
| 102 | +You may also want to provide a default priority, to be used if mapped one is not available: |
| 103 | +[source,properties] |
| 104 | +---- |
| 105 | +jira.project-group."your-project-group".priorities.default-value=3 |
| 106 | +---- |
| 107 | + |
| 108 | +==== Issue types |
| 109 | +Use the project REST API `rest/api/2/project/{projectKeyOrId}` and inspect the `issueTypes` array. |
| 110 | +Alternatively, go to your project settings page and review the project settings summary (e.g. `https://your-jira-instance/plugins/servlet/project-config/your-project-key/summary`). |
| 111 | +Get the info from upstream/downstream projects and build a mapping in the `application.properties` file as: |
| 112 | + |
| 113 | +[source,properties] |
| 114 | +---- |
| 115 | +jira.project-group."your-project-group".issue-types.mapping."source-id"=destination-id |
| 116 | +---- |
| 117 | + |
| 118 | +==== Statuses |
| 119 | +Issue status cannot be set directly through a REST API call. Instead, to change the issue status a transition has to be applied. |
| 120 | +That's why upstream issue statuses are mapped to downstream transitions. |
| 121 | +To find the list of statuses review the result of `` |
| 122 | + |
| 123 | +To find the list of transitions available for your project see, try `/api/2/issue/{issueIdOrKey}/transitions` |
| 124 | + |
| 125 | +Note: only a single transition is going to be applied. Hence, make sure that the workflow allows to apply transitions from any state. |
| 126 | + |
| 127 | +[source,properties] |
| 128 | +---- |
| 129 | +jira.project-group."your-project-group".statuses.mapping."source-status-id"=destination-transition-id |
| 130 | +---- |
| 131 | + |
| 132 | +==== Issue link types |
| 133 | +Use the project REST API `rest/api/2/issueLinkType` and build the mappings. |
| 134 | +Get the info from upstream/downstream projects and build a mapping in the `application.properties` file as: |
| 135 | + |
| 136 | +[source,properties] |
| 137 | +---- |
| 138 | +jira.project-group."your-project-group".issue-types.mapping."source-id"=destination-id |
| 139 | +---- |
| 140 | + |
| 141 | +Note, creating sub-task is not supported by the tool, as there is no way of converting a task to a sub-task through |
| 142 | +REST at the moment. Hence instead a sub-task is created as a task, and an extra issue link is created for it of a type |
| 143 | +configured via a property: |
| 144 | + |
| 145 | +[source,properties] |
| 146 | +---- |
| 147 | +jira.project-group."your-project-group".issue-link-types.parent-link-type=destination-id |
| 148 | +---- |
| 149 | + |
| 150 | +== Recommended Downstream project configuration |
| 151 | + |
| 152 | +* Make sure the mapped users are added to the project |
| 153 | +* Assign the project users to the "viewer-only" role that does not have edit permissions to prevent accidental edits. |
| 154 | +Any updates/comments should be added to the upstream Jira tickets and synced through the tool. |
| 155 | +* Service account used by the tool for communication with the downstream Jira should be assigned to the project admin |
| 156 | +role, so it could leverage the `notifyUsers` query parameter where possible. |
| 157 | +* Disable email notifications for the project. Most likely you would need to request your Jira administrator to adjust the |
| 158 | +notification scheme for the downstream project (see https://your-jira-instance/plugins/servlet/project-config/your-project-key/notifications). |
| 159 | +If for some reason the notifications have to be kept enabled, consider adjusting the scheme so that notifications for |
| 160 | +comments/issues/transitions updates do not include current assignee or reporter. |
| 161 | +While the sync tool tries to leverage the `notifyUsers` parameter to prevent emails |
| 162 | +from being sent after the actions the tool perform, not all requests respect this parameter and emails may be sent out anyway. |
| 163 | + |
| 164 | +== Deployment |
| 165 | + |
| 166 | +Currently, the tool is deployed to an OpenShift. Deployment happens automatically when pushed to the `production` branch. |
| 167 | +See a corresponding GitHub action for details. |
| 168 | + |
| 169 | +=== Initial set up |
| 170 | + |
| 171 | +1. Add your OpenShift server url as a `OPENSHIFT_SERVER_INFRA_PROD` GitHub secret. |
| 172 | +See link:.github/workflows/deploy.yml[]. |
| 173 | +2. Create a service account on OpenShift that will be used by the GitHub deploy action. |
| 174 | ++ |
| 175 | +[NOTE] |
| 176 | +==== |
| 177 | +With recent versions of OpenShift the token may not be automatically generated for the service account, |
| 178 | +if that's so create one manually, e.g. create a token valid for a week: |
| 179 | +
|
| 180 | +---- |
| 181 | +oc create token <serviceaccountname> --duration=$((7*24))h |
| 182 | +---- |
| 183 | +
|
| 184 | +See also: |
| 185 | +
|
| 186 | +* https://docs.openshift.com/container-platform/4.11/release_notes/ocp-4-11-release-notes.html#ocp-4-11-notable-technical-changes) |
| 187 | +* https://docs.openshift.com/container-platform/4.17/authentication/bound-service-account-tokens.html#bound-sa-tokens-configuring-externally_bound-service-account-tokens |
| 188 | +==== |
| 189 | ++ |
| 190 | +Having the service account token ready, add it to the GitHub secretes as `OPENSHIFT_TOKEN_INFRA_PROD` |
| 191 | +See link:.github/workflows/deploy.yml[]. |
| 192 | ++ |
| 193 | +3. The OpenShift namespace needs to be added as a `OPENSHIFT_NAMESPACE_INFRA_PROD` GitHub secrete. |
| 194 | +See link:.github/workflows/deploy.yml[]. |
| 195 | +4. Create and fill OpenShift Config maps and secrets. |
| 196 | + These configs will depend on your `application.properties` configuration |
| 197 | + and what project groups were defined. |
| 198 | + `replicate-jira-config`:: |
| 199 | + Environment variables for the application. |
| 200 | + + |
| 201 | + Put in there whatever configuration you need for your specific cluster. |
| 202 | + `replicate-jira-secrets`:: |
| 203 | + Secret environment variables for the application. |
| 204 | + + |
| 205 | + Put in there whatever secret configuration you need for your specific cluster. |
| 206 | + + |
| 207 | + In particular, any REST API tokens, management passwords, Web Hook secrets. |
| 208 | + |
| 209 | +== Management endpoints |
| 210 | + |
| 211 | +There are management endpoints to sync particular issues/comments. |
| 212 | + |
| 213 | +NOTE: TODO: Need to make sure that POST management endpoints actually work before describing all of them here... |
| 214 | + |
| 215 | +== Local development |
| 216 | + |
| 217 | +Currently, the app is set up to work with two test projects, JIRATEST1/JIRATEST2 at https://hibernate.atlassian.net/. |
| 218 | +Create and add all necessary tokens to your local `.env` file, e.g.: |
| 219 | + |
| 220 | +---- |
| 221 | +JIRA_API_URL_HIBERNATE=https://hibernate.atlassian.net/rest/api/2 |
| 222 | +JIRA_API_USER_HIBERNATE=your-user |
| 223 | +JIRA_API_TOKEN_HIBERNATE=your-PAT |
| 224 | +
|
| 225 | +JIRA_API_URL_REDHAT=https://hibernate.atlassian.net/rest/api/2 |
| 226 | +JIRA_API_USER_REDHAT=your-user |
| 227 | +JIRA_API_TOKEN_REDHAT=your-PAT |
| 228 | +---- |
| 229 | + |
| 230 | +We also use https://smee.io/ to get web hooks delivered to the local env. |
| 231 | + |
| 232 | +[[license]] |
| 233 | +== License |
| 234 | + |
| 235 | +This project is licensed under the Apache License Version 2.0. |
0 commit comments