Keycloak

keycloak is one of the most widely used, and is perfect for this case, as there is PII to secure.

What's working ?

• All functionalities of keycloak are working

• Partial search is not working, always use exact match

Integration

This demo uses docker and docker compose to run Keycloak, Postgres, and Encryption Proxy. The plan is to install and add a number of users who will then migrate the data to the encrypted version.

Identification PII data

Table	Column	Searching	Note
user_entity	username	✅	user username
user_entity	email	✅	user email
user_entity	email_constraint	✅	link for user email
user_entity	first_name	✅	user first name
user_entity	last_name	✅	user last name
user_attribute	value	✅	custom user value

With this configuration, if written into config, it will be like this:

rules:
  sql:
    # database name
    keycloak:
      # enable compatibility mode
      # you can turn it off after migration done
      compatibility_mode: true
      scheme: public
      # turn on if you want to see an original query
      # for debugging purposes, do not use in production
      log_original_query: false
      # write rules
      write:
        # user entity table
        user_entity:
          username:
            hash: true
          email:
            hash: true
          email_constraint:
            hash: true
          first_name:
            hash: true
          last_name:
            hash: true
        # user attribute table
        user_attribute:
          value:
            hash: true

In this demo we will add new columns to the user_entity and user_attribute tables, namely the _ciphertext and _digest columns. The script can be found in the sql folder, we will do it using flyway.

Clone demo repository

git clone --depth 1 https://gitlab.com/afis.me/docs/encryption-proxy.git

Change directory to

cd encryption-proxy/usecase/keycloak

Download demo licenses

This license is valid until 2025-08-13 08:18:28 GMT

curl -o licenses.pem https://storage.googleapis.com/afis.me/licenses/encryption-proxy/demo/licenses.pem

Extracting the Keycloak demo data

We use xz to compress data, so we need to install xz first if it's not available.

For Mac, you can use brew to install xz

brew install xz

For Ubuntu, you can use apt to install xz

sudo apt install xz-utils

extract data.json.tar.xz using xz

tar -xJf data.json.tar.xz

Run docker compose

The minimal specifications that will be used for this demo, you can customize, according to your needs.

Container	CPU Limit	RAM Limit
postgres:17.4	2	1024M
encryption-proxy:latest	2	1024M
keycloak:26.1.3	4	4096M

For the first time, we point keycloak to an unencrypted postgres database, and we will add some users to it.

KC_DB_URL_HOST=postgresql docker compose up -d

Make sure all services are running properly, If successful, we will get something like this

docker logs -f keycloak-keycloak-1 

INFO  [io.quarkus] (main) Keycloak 26.1.3 on JVM (powered by Quarkus 3.15.3.1) started in 5.498s. Listening on: http://0.0.0.0:8080
INFO  [io.quarkus] (main) Profile dev activated. 

docker logs -f keycloak-encryption-proxy-1

INFO Postgres Proxy initialized., line: 49, file: src/libs/postgres/server.rs
INFO Postgres Proxy listening on 0.0.0.0:5432, module: postgres, line: 134, file: src/libs/postgres/server.rs
INFO HTTP server listening on 0.0.0.0:9191, line: 52, file: src/libs/http/server.rs

Run k6 to add users

 docker run --rm -it --net keycloak_default --memory=1g --cpus=1 \
   -v $(pwd)/k6/addUser.js:/data/addUser.js:ro \
   -v $(pwd)/data.json:/tmp/data.json:ro \
   -e VUS=100 \
   -e FILE_TEST=/tmp/data.json \
   -e BASE_URL=http://keycloak:8080 \
   -w /data grafana/k6:latest run /data/addUser.js

If successful, we will get something like this

INFO[0000] Setting up test environment...                source=console
INFO[0000] Number of users to create: 29174              source=console
     checks.........................: 100.00% 29177 out of 29177
     http_req_failed................: 0.00%   0 out of 29177
     http_reqs......................: 29177   181.433317/s
     iteration_duration.............: avg=550.05ms min=241.91ms med=513.39ms max=1.92s   p(90)=643.02ms p(95)=730.89ms
     iterations.....................: 29174   181.414662/s
     vus............................: 100     min=100            max=100
     vus_max........................: 100     min=100            max=100
running (02m40.8s), 000/100 VUs, 29174 complete and 0 interrupted iterations

Check in keycloak if the number of users is correct

 docker run --rm --net keycloak_default afisme/curl-jq curl -s -X GET "http://keycloak:8080/admin/realms/master/users/count" \
  -H "Authorization: Bearer $(docker run --rm --net keycloak_default afisme/curl-jq curl \
  -s -X POST "http://keycloak:8080/realms/master/protocol/openid-connect/token"\
    -H "Content-Type: application/x-www-form-urlencoded" \
    -d "username=admin" \
    -d "password=admin" \
    -d 'grant_type=password' \
    -d 'client_id=admin-cli' | jq -r '.access_token')" \
  -H "Content-Type: application/json" | jq

Run migration

docker run --rm -it --net keycloak_default --memory=1g --cpus=1 \
   -v $(pwd)/migration.yaml:/etc/migration.yaml:ro \
   afisme/encryption-proxy:latest migrate --config /etc/migration.yaml

Restart keycloak

keycloak caches data, we need to restart to clear the cache.

KC_DB_URL_HOST=postgresql docker compose restart keycloak 

After migrating, now try searching, the response should be Unauthorized because the username has been encrypted.

docker run --rm --net keycloak_default afisme/curl-jq curl -s -X GET "http://keycloak:8080/admin/realms/master/users?first0&max=2" \
  -H "Authorization: Bearer $(docker run --rm --net keycloak_default afisme/curl-jq curl \
  -s -X POST "http://keycloak:8080/realms/master/protocol/openid-connect/token"\
    -H "Content-Type: application/x-www-form-urlencoded" \
    -d "username=admin" \
    -d "password=admin" \
    -d 'grant_type=password' \
    -d 'client_id=admin-cli' | jq -r '.access_token')" \
  -H "Content-Type: application/json" | jq

The result will be like this:

{ "error": "HTTP 401 Unauthorized"}

Change to encryption proxy

To be able to read the encrypted data, the keyclock must be pointed to the encryption-proxy.

KC_DB_URL_HOST=encryption-proxy docker compose up keycloak -d

Check if the migration was successful, try to search with match email::

docker run --rm --net keycloak_default afisme/curl-jq \
curl -s -X GET \
"http://keycloak:8080/admin/realms/master/users?first0&max=1&exact=true&email=[e-mail]" \
  -H "Authorization: Bearer $(docker run --rm --net keycloak_default afisme/curl-jq curl \
  -s -X POST "http://keycloak:8080/realms/master/protocol/openid-connect/token"\
    -H "Content-Type: application/x-www-form-urlencoded" \
    -d "username=admin" \
    -d "password=admin" \
    -d 'grant_type=password' \
    -d 'client_id=admin-cli' | jq -r '.access_token')" \
  -H "Content-Type: application/json" | jq

The result will be like this:

[
  {
    "id": "05d4dcd7-79c9-4c70-9f6a-751dfdf21031",
    "username": "dadap_waluyo11",
    "firstName": "Dadap",
    "lastName": "Waluyo",
    "email": "[e-mail]",
    "emailVerified": true,
    "attributes": {
      "phone": [
        "0847426540788"
      ]
    },
    "createdTimestamp": 1747301902358,
    "enabled": true,
    "totp": false,
    "disableableCredentialTypes": [],
    "requiredActions": [],
    "notBefore": 0,
    "access": {
      "manageGroupMembership": true,
      "view": true,
      "mapRoles": true,
      "impersonate": true,
      "manage": true
    }
  }
]

Now the keycloak database is encrypted without having to change the source code.

Database ui using dbgate

Open http://localhost:3000

Delete all containers

To remove all containers in this demo, do it using this command:

KC_DB_URL_HOST=encryption-proxy docker compose down --volumes --remove-orphans --rmi local