Skip to content

Nightly certificate

Renewal procedure for the KEPO staging client certificate used by tms-booking-nightly. For stable/production, see Certificate.

Current certificate (as of 2026-10-15):

  • CN: luova-test.takso.taksihelsinki.fi, issuer DVV TEST Service Certificates - G2R
  • Serial: 019AB9F20389C932FF72C39068A22E
  • SHA-256: 0D:19:AE:E6:29:F1:CF:67:A6:47:D6:B7:14:7B:1B:F3:CD:0B:3B:07:F1:EA:AF:6F:CC:70:6E:D5:10:CB:06:AB
  • Expires: 25.11.2026. Start the renewal with Taksi Helsinki well before this.

Overview

Nightly (staging)
Token URL https://kepo-staging.testikela.fi/staging/authorize
Invoke URL https://kepo-staging.testikela.fi/esb/api/v2/gateway/invoke
Certificate registration URL https://kepo-staging.testikela.fi/esb/registration/api/v2/account/certificates/upload
Vault server https://vault-staging.aholagroup.com
Vault secret kv-v2-nightly/tms-booking (UI), JSON fields kela_takso.cert and kela_takso.key
Servers att-nightly-tms-backend-1b-nightly, att-nightly-tms-backend-2b-nightly (GCP project att-nightly)
Supervisor program tms-booking-nightly.attracs.com

Only the token URL has the /staging/ prefix. All /esb/... paths are the same as in production; only the host differs.

tms-booking nightly runs with -vault-config and reads its whole config from Vault at startup. The consul-template file attracs-application-configs/application-configs/tms-booking/attracs-nightly.yml.ctmpl is not the source of the live nightly config.

Kela uses the certificate for two things (see TMS: KEPO):

  1. Kela's firewall only accepts connections that present a valid client certificate (and come from an allowed IP).
  2. The certificate must be registered in KEPO. A new certificate can be registered while the old one is still valid.

Renewal steps

1. Prepare the files

Taksi Helsinki delivers the new certificate from DVV as a zip containing a .cer and a .crt, plus the private key generated with the CSR.

Please note!: The .crt in the DVV zip is DER (binary). Use the .cer, which is the same certificate in PEM format. tms-booking (Go tls.X509KeyPair) and curl only accept PEM. OpenSSL 3 on a laptop auto-detects DER, so the problem only shows up on the servers (OpenSSL 1.1.1).

Check the files without printing their contents:

file *.cer *.crt *.key                     # .cer should be "PEM certificate"
openssl x509 -in <new>.cer -noout -subject -serial -enddate -fingerprint -sha256
# key matches cert when both hashes are equal:
openssl x509 -in <new>.cer -noout -pubkey | sha256sum
openssl pkey -in <new>.key -pubout | sha256sum

2. Register the certificate in KEPO staging

Run from a server whose egress IP Kela allows (currently att-nightly-tms-backend-2b-nightly, see Troubleshooting).

  1. Create a private working directory: umask 077; mkdir -m 700 ~/takso-renewal
  2. Copy the files there with these names:
    • new certificate (PEM, the .cer) → kela-cert.crt
    • new private key → kela-key.crt
    • Kela Root CA (Vault kela_takso.root_ca) → kela-root-ca.crt
    • the script below → upload_takso_cert_nightly.sh
  3. Test first. --check only fetches a token and uploads nothing:
KEPO_CLIENT_ID=<takso client_id from Vault> ./upload_takso_cert_nightly.sh --check
  1. Register:
KEPO_CLIENT_ID=<takso client_id from Vault> ./upload_takso_cert_nightly.sh

The script prompts for the client secret (hidden input). Success is HTTP 201 with a JSON body containing serialNumber (the certificate serial in decimal), notBefore, notAfter and registered.

upload_takso_cert_nightly.sh

#!/usr/bin/env bash
#
# Register a (new) TAKSO client certificate in Kela KEPO.
# Based on upload_takso_cert_no_jq_fixed_v6.sh (see Certificate.md),
# fixed for nightly / KEPO staging.
#
# Usage (run on a host whose egress IP Kela allows):
#   KEPO_CLIENT_ID=... ./upload_takso_cert_nightly.sh --check   # token only
#   KEPO_CLIENT_ID=... ./upload_takso_cert_nightly.sh           # register
#
# Environment (all optional except credentials, which are prompted if unset):
#   KEPO_ENV            staging (default) | production
#   KEPO_CLIENT_ID      KEPO client id
#   KEPO_CLIENT_SECRET  KEPO client secret
#   CLIENT_CERT         default kela-cert.crt     (the NEW cert, PEM)
#   CLIENT_KEY          default kela-key.crt      (the NEW key, PEM)
#   CA_BUNDLE           default kela-root-ca.crt
#   UPLOAD_PEM          default = CLIENT_CERT     (cert to register)
#   UPLOAD_MSG          default "Taksi Helsinki TAKSO nightly <cert serial>"
#   KEPO_UPLOAD_URL     override upload endpoint
#   KEPO_INSECURE=1     skip server TLS verification (matches tms-booking behaviour)

set -euo pipefail

KEPO_ENV="${KEPO_ENV:-staging}"
case "$KEPO_ENV" in
    staging)
        CONNECTION_URL="https://kepo-staging.testikela.fi"
        TOKEN_URL="${CONNECTION_URL}/staging/authorize"
        ;;
    production)
        CONNECTION_URL="https://kepo.kela.fi"
        TOKEN_URL="${CONNECTION_URL}/authorize"
        ;;
    *)
        echo "ERROR: KEPO_ENV must be 'staging' or 'production'" >&2
        exit 2
        ;;
esac
UPLOAD_URL="${KEPO_UPLOAD_URL:-${CONNECTION_URL}/esb/registration/api/v2/account/certificates/upload}"

CLIENT_CERT="${CLIENT_CERT:-kela-cert.crt}"
CLIENT_KEY="${CLIENT_KEY:-kela-key.crt}"
CA_BUNDLE="${CA_BUNDLE:-kela-root-ca.crt}"
UPLOAD_PEM="${UPLOAD_PEM:-$CLIENT_CERT}"

CHECK_ONLY=0
if [[ "${1:-}" == "--check" ]]; then
    CHECK_ONLY=1
elif [[ -n "${1:-}" ]]; then
    echo "Usage: $0 [--check]" >&2
    exit 2
fi

TLS_OPTS=(--cert "$CLIENT_CERT" --key "$CLIENT_KEY" --cacert "$CA_BUNDLE")
if [[ "${KEPO_INSECURE:-0}" == "1" ]]; then
    TLS_OPTS+=(-k)
fi
CURL_OPTS=(--silent --show-error --connect-timeout 10 --max-time 30)

# 0) Pre-flight checks
for f in "$CLIENT_CERT" "$CLIENT_KEY" "$CA_BUNDLE" "$UPLOAD_PEM"; do
    [[ -r "$f" ]] || { echo "ERROR: cannot read $f" >&2; exit 1; }
done

openssl x509 -in "$CLIENT_CERT" -noout >/dev/null 2>&1 \
    || { echo "ERROR: $CLIENT_CERT is not a valid PEM certificate" >&2; exit 1; }

if ! openssl x509 -in "$CLIENT_CERT" -noout -checkend 0 >/dev/null; then
    echo "ERROR: $CLIENT_CERT has expired:" >&2
    openssl x509 -in "$CLIENT_CERT" -noout -enddate >&2
    exit 1
fi

cert_pub=$(openssl x509 -in "$CLIENT_CERT" -noout -pubkey | sha256sum | cut -d' ' -f1)
key_pub=$(openssl pkey -in "$CLIENT_KEY" -pubout | sha256sum | cut -d' ' -f1)
if [[ "$cert_pub" != "$key_pub" ]]; then
    echo "ERROR: $CLIENT_KEY does not match $CLIENT_CERT" >&2
    exit 1
fi

serial=$(openssl x509 -in "$UPLOAD_PEM" -noout -serial | cut -d= -f2)
UPLOAD_MSG="${UPLOAD_MSG:-Taksi Helsinki TAKSO nightly ${serial}}"

echo "KEPO environment : $KEPO_ENV"
echo "Token URL        : $TOKEN_URL"
[[ $CHECK_ONLY -eq 1 ]] || echo "Upload URL       : $UPLOAD_URL"
echo "Client cert      : $(openssl x509 -in "$CLIENT_CERT" -noout -subject)"
echo "                   serial=$(openssl x509 -in "$CLIENT_CERT" -noout -serial | cut -d= -f2)"
echo "                   $(openssl x509 -in "$CLIENT_CERT" -noout -enddate)"
echo "                   $(openssl x509 -in "$CLIENT_CERT" -noout -fingerprint -sha256)"
echo "Key matches cert : yes"
[[ "${KEPO_INSECURE:-0}" == "1" ]] && echo "Server TLS verify: DISABLED (KEPO_INSECURE=1)"
echo

# Credentials (never hard-coded, never echoed)
if [[ -z "${KEPO_CLIENT_ID:-}" ]]; then
    read -r -p "KEPO client id: " KEPO_CLIENT_ID
fi
if [[ -z "${KEPO_CLIENT_SECRET:-}" ]]; then
    read -r -s -p "KEPO client secret: " KEPO_CLIENT_SECRET
    echo
fi

# 1) Obtain OAuth2 access token (client_credentials) over mTLS
echo "Requesting access token…"
token_resp_file=$(mktemp)
trap 'rm -f "$token_resp_file"' EXIT

http_code=$(curl "${CURL_OPTS[@]}" "${TLS_OPTS[@]}" \
    -o "$token_resp_file" -w '%{http_code}' \
    -X POST "$TOKEN_URL" \
    --data-urlencode "grant_type=client_credentials" \
    --data-urlencode "client_id=${KEPO_CLIENT_ID}" \
    --data-urlencode "client_secret=${KEPO_CLIENT_SECRET}") || {
    rc=$?
    echo "ERROR: token request failed at transport level (curl exit $rc)." >&2
    echo "       Connection reset / timeout here usually means Kela's firewall rejected" >&2
    echo "       the client certificate or the source IP. curl 60 = server cert not" >&2
    echo "       verifiable with $CA_BUNDLE (retry with KEPO_INSECURE=1)." >&2
    exit 1
}

resp=$(<"$token_resp_file")
ACCESS_TOKEN="${resp#*\"access_token\":\"}"
ACCESS_TOKEN="${ACCESS_TOKEN%%\"*}"

if [[ "$http_code" != 2* || -z "$ACCESS_TOKEN" || "$ACCESS_TOKEN" == "$resp" ]]; then
    echo "ERROR: failed to get access token (HTTP $http_code). Response:" >&2
    echo "$resp" >&2
    exit 1
fi
echo "Access token acquired (HTTP $http_code, ${#ACCESS_TOKEN} chars)."

if [[ $CHECK_ONLY -eq 1 ]]; then
    echo "--check: Kela accepted the client certificate. Nothing uploaded."
    exit 0
fi

# 2) Upload certificate (DER, base64, no line wraps) to KEPO registration API
X509_B64=$(openssl x509 -in "$UPLOAD_PEM" -outform der | base64 -w0)
JSON_PAYLOAD=$(printf '{"x509":"%s","msg":"%s"}' "$X509_B64" "$UPLOAD_MSG")

echo "Uploading certificate ($UPLOAD_PEM, serial $serial) to KEPO…"
upload_resp_file=$(mktemp)
trap 'rm -f "$token_resp_file" "$upload_resp_file"' EXIT

http_code=$(curl "${CURL_OPTS[@]}" "${TLS_OPTS[@]}" \
    -o "$upload_resp_file" -w '%{http_code}' \
    -X POST "$UPLOAD_URL" \
    -H "Authorization: Bearer ${ACCESS_TOKEN}" \
    -H "Content-Type: application/json" \
    -H "Accept: application/json" \
    --data "$JSON_PAYLOAD")

echo "HTTP $http_code"
cat "$upload_resp_file"
echo

if [[ "$http_code" != 2* ]]; then
    echo "ERROR: upload failed." >&2
    exit 1
fi
echo "Certificate registered."

3. Update Vault

  1. Open vault-staging.aholagroup.com → kv-v2-nightly → tms-booking. Write down the current version number (rollback).
  2. Create a new version in the JSON editor and replace only kela_takso.cert and kela_takso.key. Do not touch client_id, client_secret, root_ca or the URLs.

Please note!: Cert and key must be one-line JSON strings with \n escapes ("-----BEGIN CERTIFICATE-----\nMII...\n-----END CERTIFICATE-----\n"). Convert locally without printing the contents (copies straight to clipboard, includes the surrounding quotes):

python3 -I -c 'import json,sys;sys.stdout.write(json.dumps(open(sys.argv[1]).read()))' <new>.cer | wl-copy
python3 -I -c 'import json,sys;sys.stdout.write(json.dumps(open(sys.argv[1]).read()))' <new>.key | wl-copy

Rollback: create a new version from the previous version number in the Vault UI and restart again.

4. Restart tms-booking nightly

One server at a time:

# att-nightly-tms-backend-1b-nightly, then att-nightly-tms-backend-2b-nightly
sudo supervisorctl restart tms-booking-nightly.attracs.com
sudo supervisorctl status tms-booking-nightly.attracs.com

The stdout log (/tmp/tms-booking-nightly.attracs.com-stdout---supervisor-*.log) should show [config.vault] loading lines and [kela-kepo]: kela kepo client initialized.

5. Verify

Do a TAKSO search in https://tms-booking-nightly.attracs.com/kela-takso?provider=kepo and check the logs in GCP Logging (project att-nightly):

logName="projects/att-nightly/logs/platform-trace-nightly"
labels.app="tms-booking"
labels.http_url="/api/v1/integrations/kela_takso"

POST 200 /api/v1/integrations/kela_takso = OK. Check labels.server_id to see which backend served the request. (/overlap, /search and /download-history do not call Kela.)

6. Clean up

Delete the working directory on the server (shred -u ~/takso-renewal/kela-*.crt && rm -rf ~/takso-renewal) and any local one-line/exported copies of the key.

Troubleshooting

Errors appear in details.backend_error of POST 400 /api/v1/integrations/kela_takso: failed to request takso data:

Error at /staging/authorize Meaning
read: connection reset by peer TCP connects but Kela's firewall rejects the TLS handshake: client certificate expired, invalid or not PEM.
context deadline exceeded (Client.Timeout exceeded while awaiting headers) No TCP connection at all: the server's egress IP is not allowed by Kela. Certificate changes do not help.

Test the TCP connection from a server (no credentials needed):

timeout 5 bash -c 'exec 3<>/dev/tcp/91.202.114.26/443' && echo OPEN || echo NO-CONNECT

Allowed egress IPs (as of 2026-10-15):

  • 35.228.162.53 (att-nightly-tms-backend-2b-nightly): allowed
  • 35.228.37.29 (att-nightly-tms-backend-1b-nightly): not allowed, requested from KEPO support. Until then TAKSO searches that the load balancer sends to 1b time out.

History

  • 2025-10-22: nightly certificate (serial 0192B368…) expired. A new certificate was delivered on 2025-11-25 but was never put into nightly Vault or registered in KEPO staging.
  • 2026-10-15: nightly TAKSO searches failing (since at least September). New certificate registered in KEPO staging, nightly Vault updated, tms-booking restarted. 2b OK; 1b blocked by Kela's IP allowlist.