# Jira Notes

# Quick Reference:

* RDS instance is `jira2`, but database is just `jira`
* Started by systemd / `/etc/rc.d/init.d/jira`
* Code/binary in `/opt/atlassian/jira`
    - `JIRA_HOME` set in `/opt/atlassian/jira/atlassian-jira/WEB-INF/classes/jira-application.properties`
* Data in `/opt/jira-data/jira` [ JIRA_HOME ]
    - DB Config in `/opt/jira-data/jira/dbconfig.xml`
    - Attachments in `/opt/jira/data/jira/attachments`

# TLS Setup for RDS

```
# for commercial
curl https://s3.amazonaws.com/rds-downloads/rds-ca-2019-root.pem
# For govcloud
curl https://s3.us-gov-west-1.amazonaws.com/rds-downloads/rds-ca-us-gov-east-1-2017-root.pem -o root.crt
```

Place output in /home/jira/.postgresql/root.crt

```
vim /opt/jira-data/jira/dbconfig.xml
# Add ?sslmode=verify-full to the <url>
```


~# TLS Setup for RDS - OUTDATED!!!
~
~First need to update `dbconfig.xml` to tell it to use TLS and what root certs to use:
~
~```
~    <url><![CDATA[jdbc:postgresql://jira.cm5pc4cb8hlj.us-east-1.rds.amazonaws.com:5432/jira?sslmode=verify-full&sslrootcert=/opt/atlassian/jira/rds-root-chain.pem]]></url>
~
~```
~
~Then in `/opt/atlassian/jira/rds-root-chain.pem` you need the root cert(s) for RDS.  Use something like this:
~
~```
~#!/bin/bash
~
~URLS="https://s3.amazonaws.com/rds-downloads/rds-ca-2019-root.pem"
~URLS="${URLS} https://s3.amazonaws.com/rds-downloads/rds-ca-2015-root.pem"
~URLS="${URLS} https://s3-us-gov-west-1.amazonaws.com/rds-downloads/rds-ca-us-gov-east-1-2017-root.pem"
~URLS="${URLS} https://s3-us-gov-west-1.amazonaws.com/rds-downloads/rds-ca-us-gov-west-1-2017-root.pem"
~
~rm rds-root-chain.pem
~
~for i in $URLS; do
~        echo "# `basename $i`"
~        curl -s $i
~done >> rds-root-chain.pem
~
~
~```
~see [https://docs.aws.amazon.com/AmazonRDS/latest/UserGuide/UsingWithRDS.SSL.html]()
~
~There is mention of ways with newer versions of the PostgreSQL JDBC driver to use the 
~standard Java keystore for root certs.  This does not work with the version of the JDBC 
~driver skipping with Jira version 7.13, as the class needed is missing.  (There's no
~DefaultJavaSSLFactory in `postresql-9.4.1212.jar`)

One handy trick:

```
openssl s_client -starttls postgres -connect my.postgres.host:5432 # etc...
```


# Proxy setup

In ~`JIRA_HOME/bin/setenv.sh`~ `<jira binaries directory>/bin/setenv.sh`

```
JVM_SUPPORT_RECOMMENDED_ARGS=" -Dhttp.proxyHost=proxy.msoc.defpoint.local -Dhttp.proxyPort=80 -Dhttps.proxyHost=proxy.msoc.defpoint.local -Dhttps.proxyPort=80 -Dhttp.nonProxyHosts='*.defpoint.local|localhost|127.0.0.1|169.254.169.254|*.amazonaws.com'"
```

Without this, JIRA cannot download new plugins and things from the Atlassian repositories.

# Okta stuff

Okta appears to have provided their own SAML implementation for JIRA.  Which is weird, I
expected JIRA to have their own.

[https://saml-doc.okta.com/Provisioning_Docs/Okta_Jira_Authenticator_Configuration_Guide.html]()

There's a config file in `/opt/atlassian/jira/atlassian-jira/WEB-INF/classes/seraph-config.xml`
that refers to another config file `/opt/docker/okta-config-jira.xml`.  That is where the actual
SAML magic is stored.

# Load Balancer Stuff

There's stuff in web.xml that tells it that it's in front of a load balancer.  The 
proxyName and proxyPort settings matter, because they will cause redirects when
you connect to the wrong name.  Note that in the current config, the load balancer 
terminates TLS and sends plain HTTP back to JIRA itself.


```
        <Connector
        port="8080"
        relaxedPathChars="[]|"
        relaxedQueryChars="[]|{}^\`&quot;&lt;&gt;"
        maxThreads="150"
        minSpareThreads="25"
        connectionTimeout="20000"
        enableLookups="false"
        maxHttpHeaderSize="8192"
        protocol="HTTP/1.1"
        useBodyEncodingForURI="true"
        redirectPort="443"
        acceptCount="100"
        disableUploadTimeout="true"
        bindOnInit="false"
        proxyName="jira.mdr-test.defpoint.com"
        proxyPort="443"
        scheme="https"
        secure="true"
    />
```


# Useful links

[https://confluence.atlassian.com/adminjiraserver085/setting-properties-and-options-on-startup-981155694.html]()
[https://confluence.atlassian.com/jirakb/change-the-base-url-of-jira-server-in-the-database-733940375.html]()


# Undockerizing

* Fix the split attachments dir
* Move attachments out to something like EFS
* Load balancer expects to connect to port 80, which is being forwarded by docker to 8080 inside the container.

# Migrating to GovCloud

## 0. Stop Jira

```
ssh prod-jira-server
sudo systemctl stop jira
sudo systemctl disable jira
```

## 1. Sync original:
```
cd
# I believe /opt/jira-data/jira/data/attachments is the correct directory, but it could be
#   /opt/jira/data/attachments
#   /opt/jira-data/jira/import/attachments
time rsync --rsync-path="sudo rsync" -avz --delete --progress prod-jira-server:/opt/atlassian tmp/atlassian
time rsync --rsync-path="sudo rsync" -avz --delete --progress prod-jira-server:/opt/jira-data tmp/jira-data

# then restore
rsync --rsync-path="sudo rsync" -avz --delete --progress tmp/atlassian/atlassian/ gc-prod-jira-server:/opt/atlassian/
rsync --rsync-path="sudo rsync" -avz --delete --progress tmp/jira-data/jira-data/ gc-prod-jira-server:/opt/jira-data/
```

## 2. Database Dump
on legacy server:
```
curl https://s3.amazonaws.com/rds-downloads/rds-ca-2019-root.pem -o root.crt
mkdir .postgresql
mv root.crt .postgresql
# Have to install pg11 from the official postgres repositories, but we'll remove it when we're done
sudo yum install https://download.postgresql.org/pub/repos/yum/11/redhat/rhel-7-x86_64/pgdg-redhat-repo-latest.noarch.rpm
sudo yum install postgresql11
# yes, yes, yes
pg_dump postgresql://jira@jira2.csskgjc1suov.us-east-1.rds.amazonaws.com:5432/jira?sslmode=verify-full | gzip > backup.psql.gz
# Enter password which you got from /opt/jira-data/jira/dbconfig.xml
```

## Restore the database

NOTE: New DB URL: jira2.csqclvntmsrg.us-gov-east-1.rds.amazonaws.com

```
scp prod-jira-server:backup.psql.gz ./restore.psql.gz
scp restore.psql.gz gc-prod-jira-server:
ssh gc-prod-jira-server
# NOTE: This url is different from above, as there's a differnet cert needed
curl https://s3.us-gov-west-1.amazonaws.com/rds-downloads/rds-ca-us-gov-east-1-2017-root.pem -o root.crt
mkdir .postgresql
mv root.crt .postgresql
# Have to install pg11 from the official postgres repositories, but we'll remove it when we're done
sudo yum install https://download.postgresql.org/pub/repos/yum/11/redhat/rhel-7-x86_64/pgdg-redhat-repo-latest.noarch.rpm
sudo yum install postgresql11
zcat restore.psql.gz | psql --set ON_ERROR_STOP=on --single-transaction postgresql://jira@jira2.csqclvntmsrg.us-gov-east-1.rds.amazonaws.com:5432/jira?sslmode=verify-full
# No errors the first time, wtf?
```

To restore a second time:
```
echo 'DROP DATABASE jira' | psql postgresql://jira@jira2.csqclvntmsrg.us-gov-east-1.rds.amazonaws.com:5432/postgres?sslmode=verify-full
echo 'CREATE DATABASE jira' | psql postgresql://jira@jira2.csqclvntmsrg.us-gov-east-1.rds.amazonaws.com:5432/postgres?sslmode=verify-full
zcat restore.psql.gz | psql --set ON_ERROR_STOP=on --single-transaction postgresql://jira@jira2.csqclvntmsrg.us-gov-east-1.rds.amazonaws.com:5432/jira?sslmode=verify-full
```

## 3. Install software

on the new server:
```
wget https://www.atlassian.com/software/jira/downloads/binary/atlassian-jira-software-8.12.1-x64.bin
sudo chmod 755 atlassian-jira-software-8.12.1-x64.bin
sudo ./atlassian-jira-software-8.12.1-x64.bin
# Yes to install fonts
# 1 for express install
# /opt/atlassian/jira (default) for install path
# /opt/jira-data/jira (NOT THE DEFAULT) for data path
# 1 for Default ports
# y to service
# Summary:
#   Installation Directory: /opt/atlassian/jira
#   Home Directory: /opt/jira-data/jira
#   HTTP Port: 8080
#   RMI Port: 8005
#   Install as service: Yes
# no to not start jira now
```

## 4. Update Configuration

```
sudo chown -R jira:jira /opt/atlassian /opt/jira-data
vim /opt/jira-data/jira/dbconfig.xml
# update the db connection string
vim /opt/atlassian/jira/conf/server.xml
# Update jira.mdr.defpoint.com to jira.xdr.accenturefederalcyber.com
vim /opt/atlassian/jira/bin/setenv.sh
# Set:
#  JVM_SUPPORT_RECOMMENDED_ARGS="-Dhttp.proxyHost=proxy.pvt.xdr.accenturefederalcyber.com -Dhttp.proxyPort=80 -Dhttps.proxyHost=proxy.pvt.xdr.accenturefederalcyber.com -Dhttps.proxyPort=80"
```

```
vim /opt/atlassian/jira/conf/okta-config-jira.xml
# copy from okta, remove the <xml> header
```

Login and resolve any issues, probably with basename
Update mail server.

NOTES:
new jira db:
jira2.csqclvntmsrg.us-gov-east-1.rds.amazonaws.com

# Upgrade Notes - 5/2021

Manual upgrade, recommended for Linux: https://confluence.atlassian.com/adminjiraserver0813/upgrading-jira-manual-1027137578.html

Tickets: https://jira.xdr.accenturefederalcyber.com/browse/MSOCI-1606
         https://jira.atlassian.com/browse/JRASERVER-71652

Current Version: v8.12.1#812002-sha1:7c28d59
Target Version:  v8.13.6 (LTS)  https://www.atlassian.com/software/jira/download

Download 'tar.gz' from https://www.atlassian.com/software/jira/download
Copy to server.

## Prep: Plenty of backups
```
sudo cat /opt/jira-data/jira/dbconfig.xml
pg_dump postgresql://jira@jira2.csqclvntmsrg.us-gov-east-1.rds.amazonaws.com:5432/jira?sslmode=verify-full | gzip > backup.202105.psql.gz
cd /opt
sudo rsync -rav atlassian atlassian.bak.202105
sudo rsync -rav jira-data jira-data.202106
```

Then in the GUI:
1) Take a snapshot of the instance (be sure to check the box to not restart it)
2) Take a snapshot of the database

## Prep: Extract and Configure

```
cd /opt/atlassian
sudo tar xvzf ~/atlassian-jira-software-8.13.6.tar.gz
sudo mv atlassian-jira-software-8.13.6-standalone jira-8.13.6
cd jira-8.13.6/
sudo chown -R jira:jira .
sudo cp ../jira/atlassian-jira/WEB-INF/lib/okta-jira-3.1.3.jar ./atlassian-jira/WEB-INF/lib/okta-jira-3.1.3.jar
sudo cp ../jira/atlassian-jira/okta_login.jsp ./atlassian-jira/okta_login.jsp
sudo cp ../jira/conf/okta-config-jira.xml ./conf/okta-config-jira.xml

sudo vim atlassian-jira/WEB-INF/classes/jira-application.properties
# set jira.home to /opt/jira-data/jira
sudo vim bin/setenv.sh
# set JIRA_HOME (maybe? It wasn't set in the old one, but i'm setting it)
# Copy JVM_SUPPORT_RECOMMENDED_ARGS from old copy
# Note: previous setting JVM_MAXIMUM_MEMORY="768m" is now JVM_MAXIMUM_MEMORY="2048m"
```

In `atlassian-jira/WEB-INF/web.xml`, just before `THIS MUST BE THE LAST FILTER IN THE DEFINED CHAIN`, add the following:
```
    <filter>
        <filter-name>OktaLoginFilter</filter-name>
        <filter-class>com.atlassian.jira.authenticator.okta.OktaLoginFilter</filter-class>
    </filter>

    <filter-mapping>
        <filter-name>OktaLoginFilter</filter-name>
        <url-pattern>/*</url-pattern>
        <dispatcher>REQUEST</dispatcher>
        <dispatcher>FORWARD</dispatcher>
    </filter-mapping>
```

In `conf/server.xml` in the first 'connector' clause, update the last two lines (only chagne on the first line is the `/>`):: 
```
acceptCount="100" disableUploadTimeout="true" bindOnInit="false"
proxyName="jira.xdr.accenturefederalcyber.com" proxyPort="443" scheme="https" secure="true"/>
```

In `atlassian-jira/WEB-INF/classes/seraph-config.xml`:

1. For `<param-name>logout.url</param-name>`, set:
```
<param-value>https://mdr-multipass.okta.com</param-value>
```

2. Comment out the following:
```
    <!-- <authenticator class="com.atlassian.jira.security.login.JiraSeraphAuthenticator"/> -->
```

3. After the last `<!-- CROWD:END -->`, add the following section (note the fixed path):
```
    <!-- MDR OKTA:START -->

        <authenticator class="com.atlassian.jira.authenticator.okta.OktaJiraAuthenticator30">
            <init-param>
                <param-name>okta.config.file</param-name>
                <param-value>/opt/atlassian/jira-8.13.6/conf/okta-config-jira.xml</param-value>
            </init-param>
        </authenticator>

    <!-- MDR OKTA:END -->
```

Update okta to latest (optional):
1. Log into okta admin
2. go to settings->downloads
3. Download the latest okta plugin
4. Copy to the server

```
ssh gc-prod-okta-server
cd /opt/atlassian/jira-8.13.6
sudo rm atlassian-jira/WEB-INF/lib/okta-jira-3.1.3.jar
sudo mv ~frederick_t_damstra/okta-jira-3.1.5.jar atlassian-jira/WEB-INF/lib/
sudo chown jira:jira atlassian-jira/WEB-INF/lib/okta-jira-3.1.5.jar
sudo chmod 644 atlassian-jira/WEB-INF/lib/okta-jira-3.1.5.jar
```

## Cutover:

```
sudo /etc/init.d/jira stop
# In GUI, take db snapshot
cd /opt
sudo rsync -rav atlassian atlassian.bak.202105
sudo rsync -rav jira-data jira-data.202106
sudo mv /opt/atlassian/jira /opt/atlassian/jira-8.12.1
vim /etc/rc.d/init.d/jira
# Update path to /opt/atlassian/jira/bin
sudo /etc/init.d/jira start
```

Note, the 'check upgrade' step, suggested that these might have been modified, too:
```
jira-application.properties
WEB-INF/web.xml
seraph-config.xml
``` 

Also, if okta doesn't work, you can try placing okta back:
```
sudo cp /opt/atlassian/jira/atlassian-jira/WEB-INF/lib/okta-jira-3.1.3.jar /opt/atlassian/jira-8.13.6/atlassian-jira/WEB-INF/lib/okta-jira-3.1.3.jar
```

## Problems during 4/27/2021 upgrade:

### Jira wouldn't start because JAVA_HOME was not yet. 
1. edit bin/setenv.sh, search for JAVA_HOME and replace with:
```
JAVA_HOME="/opt/atlassian/jre/"; export JAVA_HOME
```
2. Copy the old JRE (and I bet this needs to be updated)

### Jira started, but database won't connect. This was because of SSL.
1. temporarily removed the verify-ssl command from /opt/jira-data/jira/dbconfig.xml
2. It started.
3. Stopped it after it initialized

1. copied /home/jira/.posgres to /opt/jira-data/jira/.postgres (this contains the root cert)
no change
Tried /opt/jira-data/.postgres, too.

Left it with `sslmode=require`.

### Noisy catalina error: `The encoding [binary] is not recognised by the JRE`, with full stack trace:

see https://jira.atlassian.com/browse/JRASERVER-71265
added 3 lines to /opt/atlassian/jira-8.13.6/conf/logging.properties:
```
# per https://jira.atlassian.com/browse/JRASERVER-71265
# Surpress 'The encoding [binary] is not recognised by the JRE'
org.apache.catalina.connector.Response.level = ERROR
```
 
## If all is well

do a little dance