Improve profile support

README.md

@@ -18,6 +18,7 @@ for automating the process on remote servers.
 - [Automating updates](#automating-updates)
 - [Structure](#structure)
 - [Custom template for configuration](#custom-template-for-configuration)
+- [Configuration Variables](#configuration-variables)
 - [Server-Types](#server-types)
 - [Revoke a certificate](#revoke-a-certificate)
 - [Elliptic curve keys](#elliptic-curve-keys)
@@ -532,6 +533,76 @@ CHECK_REMOTE_WAIT="1" # wait 1 second before checking the remote server
 
 ```
 
+## Configuration Variables
+
+All variables below can be set at either the account level (`~/.getssl/getssl.cfg`) or the per-domain level (`~/.getssl/<domain>/getssl.cfg`). Settings at the domain level override those at the account level.
+
+| Variable | Default | Description |
+|----------|---------|-------------|
+| `ACCOUNT_EMAIL` | `""` | Contact email for the ACME account (used for expiry reminders, etc.) |
+| `ACCOUNT_KEY_LENGTH` | `4096` | Account key length in bits |
+| `ACCOUNT_KEY` | `"$WORKING_DIR/account.key"` | Path to the account key file |
+| `ACCOUNT_KEY_TYPE` | `"rsa"` | Account key type (`rsa`, `prime256v1`, `secp384r1`, `secp521r1`) |
+| `ACL` | | Challenge file locations for the domain and each SAN (local, `ssh:`, `ftp:`, `sftp:`, `davs:`) |
+| `ACME_RESPONSE_PENDING_WAIT` | `5` | Seconds to wait between polling when ACME status is pending/processing |
+| `AGREEMENT` | `""` | Terms-of-service agreement URL; if blank, the CA's current agreement is used |
+| `ARI_ENABLE` | `"true"` | Whether to consult the CA's ACME Renewal Information (ARI) endpoint before renewing |
+| `AUTH_DNS_SERVER` | `""` | Specific authoritative DNS server to use for challenge verification |
+| `CA` | Let's Encrypt Staging | URL of the ACME CA directory. Production: `https://acme-v02.api.letsencrypt.org` |
+| `CA_CERT_LOCATION` | `""` | Destination for the CA (chain) certificate file |
+| `CHALLENGE_CHECK_TYPE` | `"http"` | Protocol used to verify the challenge URL (`http` or `https`) |
+| `CHECK_ALL_AUTH_DNS` | `"false"` | Check the DNS challenge token on all authoritative DNS servers, not just one |
+| `CHECK_PUBLIC_DNS_SERVER` | `"true"` | Also query the public DNS server (with `VALIDATE_VIA_DNS`) |
+| `CHECK_REMOTE` | `"true"` | Check the remote server after install to confirm correct certificate is loaded |
+| `CHECK_REMOTE_WAIT` | `0` | Seconds to wait after `RELOAD_CMD` before checking the remote server |
+| `CSR_SUBJECT` | `"/"` | Subject for the CSR (most fields are stripped by Let's Encrypt) |
+| `DEACTIVATE_AUTH` | `"false"` | Deactivate authorization after each use (requires re-authorization next time) |
+| `DEFAULT_REVOKE_CA` | Let's Encrypt | Default CA server used by `getssl -r` if none is specified on the command line |
+| `DNS_ADD_COMMAND` | `""` | Script/command to add a DNS challenge TXT record |
+| `DNS_DEL_COMMAND` | `""` | Script/command to remove a DNS challenge TXT record |
+| `DNS_EXTRA_WAIT` | `0` | Seconds to wait after DNS propagation before asking the CA to validate |
+| `DNS_WAIT` | `5` | Seconds between DNS propagation re-checks |
+| `DNS_WAIT_COUNT` | `100` | Maximum number of DNS propagation re-checks before giving up |
+| `DNS_WAIT_RETRY_ADD` | `"false"` | Re-run `DNS_ADD_COMMAND` every 10 retries if DNS hasn't updated |
+| `DOMAIN_CERT_LOCATION` | `""` | Destination for the domain certificate file |
+| `DOMAIN_CHAIN_LOCATION` | `""` | Destination for a combined domain + CA certificate file |
+| `DOMAIN_KEY_CERT_LOCATION` | `""` | Destination for a combined private key + domain certificate file |
+| `DOMAIN_KEY_LENGTH` | `4096` | Domain key length in bits (RSA only) |
+| `DOMAIN_KEY_LOCATION` | `""` | Destination for the private key file |
+| `DOMAIN_PEM_LOCATION` | `""` | Destination for a combined private key + domain + CA certificate file |
+| `DOMAIN_STORAGE` | `~/.getssl` | Directory where all per-domain config and certificates are stored |
+| `DUAL_RSA_ECDSA` | `"false"` | Obtain both an RSA and an ECDSA certificate for each order |
+| `FTP_ARGS` | `""` | Extra arguments passed to `ftp` (e.g. `-p` for passive mode) |
+| `FTP_OPTIONS` | `""` | Options inserted into the ftp upload script (e.g. `passive`) |
+| `FTP_PORT` | `""` | Port used for ftp/sftp/ftps/ftpes uploads |
+| `FTPS_OPTIONS` | `""` | Options passed to `curl` for ftps/ftpes uploads (e.g. `--insecure`) |
+| `FULL_CHAIN_INCLUDE_ROOT` | `"false"` | Include the root CA certificate in the full chain file |
+| `GETSSL_IGNORE_CP_PRESERVE` | `"false"` | Don't try to preserve permissions when copying files |
+| `HTTP_TOKEN_CHECK_WAIT` | `0` | Seconds to wait after uploading a token before verifying it |
+| `IGNORE_DIRECTORY_DOMAIN` | `"false"` | Don't include the directory name as the main domain on the certificate |
+| `OCSP_MUST_STAPLE` | `"false"` | Add the OCSP Must-Staple extension to the certificate |
+| `PREFERRED_CHAIN` | `""` | Substring match against issuer CN to select a specific root chain |
+| `PREVENT_NON_INTERACTIVE_RENEWAL` | `"false"` | Disallow non-interactive (cron) reissue of this certificate |
+| `PRIVATE_KEY_ALG` | `"rsa"` | Domain key algorithm (`rsa`, `prime256v1`, `secp384r1`, `secp521r1`) |
+| `PROFILE` | `""` | ACME certificate profile name, if offered by the CA |
+| `PUBLIC_DNS_SERVER` | `""` | Public DNS server to consult alongside the authoritative servers |
+| `RELOAD_CMD` | `""` | Command(s) to reload services after installing a new certificate |
+| `REMOTE_EXTRA` | `""` | Extra `curl` options used when `SERVER_TYPE` is a port number |
+| `RENEW_ALLOW` | `30` | Days before expiry within which renewal is allowed |
+| `REUSE_PRIVATE_KEY` | `"true"` | Reuse the existing private key when renewing a certificate |
+| `SANS` | `""` | Comma-separated list of Subject Alternative Names |
+| `SCP_OPTS` | `""` | Extra options passed to `scp` (e.g. `-i identity_file`) |
+| `SERVER_TYPE` | `"https"` | Service type to check for correct certificate installation (see [Server-Types](#server-types)) |
+| `SFTP_OPTS` | `""` | Extra options passed to `sftp` (e.g. `-P 1234`) |
+| `SKIP_HTTP_TOKEN_CHECK` | `"false"` | Don't fetch the challenge URL after uploading the token |
+| `SSH_OPTS` | `""` | Extra options passed to `ssh` (e.g. `-p 1234 -i identity_file`) |
+| `SSLCONF` | `openssl.cnf` | Path to the OpenSSL configuration file |
+| `TOKEN_USER_ID` | `""` | User (and group, as `user.group`) that should own the token file |
+| `USE_SINGLE_ACL` | `"false"` | Apply the first `ACL` entry to all domains, including each SAN |
+| `VALIDATE_VIA_DNS` | `"false"` | Use the DNS-01 challenge instead of HTTP-01 |
+
+Full details of each variable, with examples, are available in the [wiki Configuration Variables](https://github.com/srvrco/getssl/wiki/Config-variables) page and in the comments of the per-domain config template.
+
 ## Server-Types
 
 OpenSSL has built-in support for getting the certificate from a number of SSL services

getssl

@@ -1227,13 +1227,9 @@ create_order() {
   done
   dstring="${dstring::${#dstring}-1}]"
   replaces="${_REPLACES:+, \"replaces\": \"${_REPLACES}\"}"
+  profile="${PROFILE:+, \"profile\": \"$PROFILE\"}"
 
-  # Check if the server supports profiles using the URL_profiles variable
-  if [[ -z "$URL_profiles" ]]; then
-    request="{\"identifiers\": $dstring$replaces}"
-  else
-    request="{\"identifiers\": $dstring, \"profile\": \"$PROFILE\"$replaces}"
-  fi
+  request="{\"identifiers\": $dstring$profile$replaces}"
   send_signed_request "$URL_newOrder" "$request"
   OrderLink=$(echo "$responseHeaders" | grep -i location | awk '{print $2}'| tr -d '\r\n ')
   debug "Order link $OrderLink"
@@ -2628,7 +2624,15 @@ obtain_ca_resource_locations()
     URL_revoke=$(echo "$ca_all_loc" | grep "revokeCert" | awk -F'"' '{print $4}')
     URL_renewInfo=$(echo "$ca_all_loc" | grep "renewalInfo" | awk -F'"' '{print $4}')
 
-    URL_profiles=""
+    if [[ -n "$URL_new_reg" ]] || [[ -n "$URL_newAccount" ]]; then
+      break
+    fi
+  done
+
+  # If a directory offers both versions, select V2.
+  if [[ -n "$URL_newAccount" ]]; then
+    API=2
+    # Only check profile support for V2
     # Check if we have a profiles element
     if echo "$ca_all_loc" | grep -q '"profiles"'; then
       meta=$(get_json_value "$ca_all_loc" "meta")
@@ -2640,23 +2644,14 @@ obtain_ca_resource_locations()
       for key in "${URL_profiles_array[@]}"; do
         debug "$key"
       done
-
-      # if the profile isn't set, then use the first value in the profile array
-      if [[ -z "$PROFILE" ]]; then
-        PROFILE=${URL_profiles_array[0]}
-      fi
-    fi
-
-    if [[ -n "$URL_new_reg" ]] || [[ -n "$URL_newAccount" ]]; then
-      break
+      debug "Chosen profile: ${PROFILE:-server default}"
+    else
+      # If we don't have profile support, then clear out PROFILE var
+      PROFILE=""
     fi
-  done
-
-  # If a directory offers both versions, select V2.
-  if [[ -n "$URL_newAccount" ]]; then
-    API=2
   elif [[ -n "$URL_new_reg" ]]; then
     API=1
+    PROFILE=""
   else
     error_exit "unknown API version"
   fi
@@ -3056,6 +3051,9 @@ write_domain_template() { # write out a template file for a domain.
 		# This server issues full certificates, however has rate limits
 		#CA="https://acme-v02.api.letsencrypt.org"
 
+		# Certificate profile to use.  If left blank then use the server default
+		#PROFILE=""
+
 		# Private key types - can be rsa, prime256v1, secp384r1 or secp521r1
 		#PRIVATE_KEY_ALG="rsa"
 
@@ -3141,6 +3139,9 @@ write_getssl_template() { # write out the main template file
 	# This server issues full certificates, however has rate limits
 	#CA="https://acme-v02.api.letsencrypt.org"
 
+	# Certificate profile to use. If left blank then use the server default
+	#PROFILE=""
+
 	# The agreement that must be signed with the CA, if not defined the default agreement will be used
 	#AGREEMENT="$AGREEMENT"
 

test/19-test-add-to-sans.bats

@@ -34,7 +34,7 @@ teardown_file() {
     if [ -n "$STAGING" ]; then
         skip "Not trying on staging server yet"
     fi
-    CONFIG_FILE="getssl-dns01.cfg"
+    CONFIG_FILE="getssl-dns01-no-ari.cfg"
     setup_environment
     init_getssl
 
@@ -48,7 +48,7 @@ teardown_file() {
     if [ -n "$STAGING" ]; then
         skip "Not trying on staging server yet"
     fi
-    CONFIG_FILE="getssl-dns01.cfg"
+    CONFIG_FILE="getssl-dns01-no-ari.cfg"
 
     . "${CODE_DIR}/test/test-config/${CONFIG_FILE}"
     CERT=${INSTALL_DIR}/.getssl/${GETSSL_CMD_HOST}/${GETSSL_CMD_HOST}.crt
@@ -78,7 +78,7 @@ teardown_file() {
     if [ -n "$STAGING" ]; then
         skip "Not trying on staging server yet"
     fi
-    CONFIG_FILE="getssl-dns01.cfg"
+    CONFIG_FILE="getssl-dns01-no-ari.cfg"
 
     cat <<- EOF > ${INSTALL_DIR}/.getssl/${GETSSL_CMD_HOST}/getssl_test_specific.cfg
 SANS="a.${GETSSL_HOST}"
@@ -112,7 +112,7 @@ EOF
     if [ -n "$STAGING" ]; then
         skip "Not trying on staging server yet"
     fi
-    CONFIG_FILE="getssl-dns01.cfg"
+    CONFIG_FILE="getssl-dns01-no-ari.cfg"
 
     cat <<- EOF > ${INSTALL_DIR}/.getssl/${GETSSL_CMD_HOST}/getssl_test_specific.cfg
 SANS="a.${GETSSL_HOST}"
@@ -146,7 +146,7 @@ EOF
     if [ -n "$STAGING" ]; then
         skip "Not trying on staging server yet"
     fi
-    CONFIG_FILE="getssl-dns01.cfg"
+    CONFIG_FILE="getssl-dns01-no-ari.cfg"
 
     cat <<- EOF > ${INSTALL_DIR}/.getssl/${GETSSL_CMD_HOST}/getssl_test_specific.cfg
 IGNORE_DIRECTORY_DOMAIN="true"

test/20-wildcard-simple.bats

@@ -19,7 +19,7 @@ setup() {
 
 
 @test "Create wildcard certificate" {
-    CONFIG_FILE="getssl-dns01.cfg"
+    CONFIG_FILE="getssl-dns01-no-ari.cfg"
 
     GETSSL_CMD_HOST="*.${GETSSL_HOST}"
     setup_environment

test/7-test-renewal.bats

@@ -37,7 +37,7 @@ teardown_file() {
     if [ -n "$STAGING" ]; then
         skip "Not testing renewal on staging server"
     fi
-    CONFIG_FILE="getssl-dns01.cfg"
+    CONFIG_FILE="getssl-dns01-no-ari.cfg"
     setup_environment
     init_getssl
 
@@ -52,7 +52,7 @@ teardown_file() {
         skip "Not trying on staging server yet"
     fi
 
-    CONFIG_FILE="getssl-dns01.cfg"
+    CONFIG_FILE="getssl-dns01-no-ari.cfg"
     . "${CODE_DIR}/test/test-config/${CONFIG_FILE}"
     CERT=${INSTALL_DIR}/.getssl/${GETSSL_CMD_HOST}/${GETSSL_CMD_HOST}.crt
     ORIGINAL_ENDDATE=$(openssl x509 -in "$CERT" -noout -enddate 2>/dev/null| cut -d= -f 2-)
@@ -81,7 +81,7 @@ teardown_file() {
         skip "Not trying on staging server yet"
     fi
 
-    CONFIG_FILE="getssl-dns01.cfg"
+    CONFIG_FILE="getssl-dns01-no-ari.cfg"
     cat <<- EOF > ${INSTALL_DIR}/.getssl/${GETSSL_CMD_HOST}/getssl_test_specific.cfg
 RENEW_ALLOW=2000
 EOF

test/test-config/getssl-dns01-dual-rsa-ecdsa-old-nginx.cfg

@@ -10,6 +10,7 @@ DNS_DEL_COMMAND="/getssl/dns_scripts/dns_del_challtestsrv"
 AUTH_DNS_SERVER=10.30.50.3
 DNS_EXTRA_WAIT=0
 ARI_ENABLE="false"
+PROFILE="default"
 
 DUAL_RSA_ECDSA="true"
 ACCOUNT_KEY_TYPE="prime256v1"

test/test-config/getssl-dns01-no-ari.cfg

@@ -0,0 +1,62 @@
+# Test that the script works with dns
+
+VALIDATE_VIA_DNS=true
+# Speed up the test by reducing the number or retries and the wait between retries.
+DNS_WAIT=2
+DNS_WAIT_COUNT=11
+DNS_EXTRA_WAIT=0
+ARI_ENABLE="false"
+PROFILE="default"
+
+if [ -z "$STAGING" ]; then
+    # Settings for challtestserv dns provider running in local docker
+    CA="https://pebble:14000/dir"
+
+    DNS_ADD_COMMAND="/getssl/dns_scripts/dns_add_challtestsrv"
+    DNS_DEL_COMMAND="/getssl/dns_scripts/dns_del_challtestsrv"
+    AUTH_DNS_SERVER=10.30.50.3
+else
+    # Settings for external dns provider and staging server
+    CA="https://acme-staging-v02.api.letsencrypt.org/directory"
+
+    # Re-use the account key when calling the staging server (otherwise hit limits)
+    ACCOUNT_KEY="${HOME}/account.key"
+    DEACTIVATE_AUTH="true"
+
+    DNS_ADD_COMMAND="/getssl/dns_scripts/dns_add_${dynamic_dns}"
+    DNS_DEL_COMMAND="/getssl/dns_scripts/dns_del_${dynamic_dns}"
+    PUBLIC_DNS_SERVER="8.8.8.8" # resolver1.infoserve.de"
+    if [[ "${dynamic_dns}" == "dynu" ]]; then
+        AUTH_DNS_SERVER=ns1.dynu.com
+    elif [[ "${dynamic_dns}" != "acmedns" ]]; then
+        AUTH_DNS_SERVER=ns1.duckdns.org
+    fi
+    CHECK_ALL_AUTH_DNS="true"
+    CHECK_PUBLIC_DNS_SERVER="true"
+    if [[ "${dynamic_dns}" != "acmedns" ]]; then
+        DNS_WAIT=30
+        DNS_WAIT_COUNT=20
+        DNS_EXTRA_WAIT=120
+    fi
+fi
+# Additional domains - this could be multiple domains / subdomains in a comma separated list
+SANS=""
+
+# Location for all your certs, these can either be on the server (full path name)
+# or using ssh /sftp as for the ACL
+DOMAIN_CERT_LOCATION="/etc/nginx/pki/server.crt"
+DOMAIN_KEY_LOCATION="/etc/nginx/pki/private/server.key"
+CA_CERT_LOCATION="/etc/nginx/pki/chain.crt"
+DOMAIN_CHAIN_LOCATION="" # this is the domain cert and CA cert
+DOMAIN_PEM_LOCATION="" # this is the domain_key, domain cert and CA cert
+
+# The command needed to reload apache / nginx or whatever you use
+RELOAD_CMD="cp /getssl/test/test-config/nginx-ubuntu-ssl ${NGINX_CONFIG} && /getssl/test/restart-nginx"
+
+# Define the server type and confirm correct certificate is installed
+SERVER_TYPE="https"
+CHECK_REMOTE="true"
+
+if [[ -s "$DOMAIN_DIR/getssl_test_specific.cfg" ]]; then
+    . $DOMAIN_DIR/getssl_test_specific.cfg
+fi

test/test-config/getssl-dns01.cfg

@@ -5,7 +5,6 @@ VALIDATE_VIA_DNS=true
 DNS_WAIT=2
 DNS_WAIT_COUNT=11
 DNS_EXTRA_WAIT=0
-ARI_ENABLE="false"
 
 if [ -z "$STAGING" ]; then
     # Settings for challtestserv dns provider running in local docker

test/test-config/getssl-http01-dual-rsa-ecdsa-old-nginx.cfg

@@ -8,6 +8,7 @@ DUAL_RSA_ECDSA="true"
 ACCOUNT_KEY_TYPE="prime256v1"
 PRIVATE_KEY_ALG="prime256v1"
 ARI_ENABLE="false"
+PROFILE="default"
 
 # Additional domains - this could be multiple domains / subdomains in a comma separated list
 SANS=""

test/test-config/getssl-http01-no-ari.cfg

@@ -14,6 +14,7 @@ ACL=('/var/www/html/.well-known/acme-challenge')
 USE_SINGLE_ACL="false"
 
 ARI_ENABLE="false"
+PROFILE="default"
 
 # Location for all your certs, these can either be on the server (full path name)
 # or using ssh /sftp as for the ACL