Kafka Docker Images for other CPU architectures

08.02.2022, Michael Gerber

Required Packages

The following packages are required to build the Kafka Docker images:

• socat
• docker
• java
• maven

Run Docker Builds from Maven

The Kafka Docker build is based on the Maven plugin dockerfile-maven-plugin. Unfortunately, at least on an M1 Mac, it did not work out of the box. Maven builds failed with the following error:

[ERROR] Failed to execute goal com.spotify:dockerfile-maven-plugin:1.4.13:build (package) on project cp-base-new: Could not build image: java.util.concurrent.ExecutionException: com.spotify.docker.client.shaded.javax.ws.rs.ProcessingException: java.lang.UnsatisfiedLinkError: could not load FFI provider com.spotify.docker.client.shaded.jnr.ffi.provider.jffi.Provider: ExceptionInInitializerError: Can't overwrite cause with java.lang.UnsatisfiedLinkError: java.lang.UnsatisfiedLinkError: Can't load library: /var/folders/90/trptjc/T/jffi6971633642015255287.dylib

As a workaround, the Docker control socket can be exposed on the TCP port 2375. This can be achieved with socat:

socat TCP-LISTEN:2375,reuseaddr,fork UNIX-CONNECT:/var/run/docker.sock

Now update the environment variable DOCKER_HOST to point to tcp://127.0.0.1:2375:

export DOCKER_HOST=tcp://127.0.0.1:2375

Build the Base Image

1. Clone the common-docker git repository

2. Check out the desired version git checkout tags/v7.0.0

3. Open the pom.xml file from the root directory and add the confluent repository:

    <repositories>
        <repository>
            <id>confluent</id>
            <url>https://packages.confluent.io/maven/</url>
        </repository>
    </repositories>
   

4. Run the following command to see all available yum packages and versions:

   docker run --rm -ti confluentinc/cp-base-new yum list available
   

5. Change the property ubi.openssl.version in the pom.xml to the latest available version (check the website for the latest version)

6. Change the property ubi.zulu.openjdk.version in the pom.xml to the latest available version (check the website for the latest version)

7. Run the following command to build the base image:

   mvn clean install \
     -DskipTests -Pdocker \
     -Ddocker.registry=nxt/ -Ddocker.skip-test=true
   

Build Kafka and Zookeeper Images

1. Clone the kafka-images git repository

2. Check out the desired version git checkout tags/v7.0.0

3. Run the following command to build the Kafka and Zookeeper image:

   mvn clean package \
     -DskipTests -Pdocker \
     -DCONFLUENT_PACKAGES_REPO='https://packages.confluent.io/rpm/7.0' \
     -DCONFLUENT_VERSION=7.0.0 \
     -Ddocker.registry=nxt/
   

Build Schema Registry Image

1. Clone the schema-registry-images git repository

2. Check out the desired version git checkout tags/v7.0.0

3. Run the following command to build the Schema Registry image:

   mvn clean package \
     -DskipTests -Pdocker \
     -DCONFLUENT_PACKAGES_REPO='https://packages.confluent.io/rpm/7.0' \
     -DCONFLUENT_VERSION=7.0.0 \
     -Ddocker.registry=nxt/
   

Verify the new Images

Run docker images | grep confluentinc to list all images:

$ docker images | grep confluentinc
nxt/confluentinc/cp-server-connect        7.0.0-ubi8               ccee5c62b5e7   2 minutes ago       2.13GB
nxt/confluentinc/cp-server-connect-base   7.0.0-ubi8               b8f607a05437   2 minutes ago       2.13GB
nxt/confluentinc/cp-kafka-connect         7.0.0-ubi8               a639af003ccc   3 minutes ago       1.42GB
nxt/confluentinc/cp-schema-registry       7.0.0-ubi8               d4ee053964d5   4 minutes ago       1.68GB
nxt/confluentinc/cp-kafka-connect-base    7.0.0-ubi8               596428b5c5c1   4 minutes ago       1.42GB
nxt/confluentinc/cp-enterprise-kafka      7.0.0-ubi8               9cbafe6c63ca   5 minutes ago       957MB
nxt/confluentinc/cp-kafka                 7.0.0-ubi8               d86c6260fd28   6 minutes ago       816MB
nxt/confluentinc/cp-server                7.0.0-ubi8               e5fca4aba934   7 minutes ago       1.58GB
nxt/confluentinc/cp-zookeeper             7.0.0-ubi8               b6bbc042ea65   8 minutes ago       816MB
nxt/confluentinc/cp-jmxterm               7.0.0-ubi8               6e3350e640a1   13 minutes ago      716MB
nxt/confluentinc/cp-base-new              7.0.0-ubi8               7e2fb9b02b52   14 minutes ago      708MB