Add GizmoSQL

[prmoore77]

Closes #733

Thank You for Your Contribution!

We appreciate your effort and contribution to the project. To ensure that your Pull Request (PR) adheres to our guidelines, please ensure to review the rules mentioned in our contribution guidelines:

ClickHouse/ClickBench Contribution Rules

Thank you for your attention to these details and for helping us maintain the quality and integrity of the project.

[CLAassistant]

All committers have signed the CLA.

[george-larionov]

Hi @prmoore77, thanks for your submission! However, I am having trouble getting it to run. When I run benchmark.sh I get the following output when creating the table (line 33):

Jan 10, 2026 6:18:08 PM org.apache.arrow.driver.jdbc.shaded.org.apache.arrow.flight.auth2.ClientHandshakeWrapper doClientHandshake
SEVERE: Failed on completing future
org.apache.arrow.driver.jdbc.shaded.org.apache.arrow.flight.FlightRuntimeException: UNAVAILABLE: io exception
        at org.apache.arrow.driver.jdbc.shaded.org.apache.arrow.flight.CallStatus.toRuntimeException(CallStatus.java:121)
        at org.apache.arrow.driver.jdbc.shaded.org.apache.arrow.flight.grpc.StatusUtils.fromGrpcRuntimeException(StatusUtils.java:161)
        at org.apache.arrow.driver.jdbc.shaded.org.apache.arrow.flight.grpc.StatusUtils.fromThrowable(StatusUtils.java:182)
        at org.apache.arrow.driver.jdbc.shaded.org.apache.arrow.flight.auth2.ClientHandshakeWrapper.doClientHandshake(ClientHandshakeWrapper.java:55)
        at org.apache.arrow.driver.jdbc.shaded.org.apache.arrow.flight.FlightClient.handshake(FlightClient.java:202)
        at org.apache.arrow.driver.jdbc.client.utils.ClientAuthenticationUtils.getAuthenticate(ClientAuthenticationUtils.java:107)
        at org.apache.arrow.driver.jdbc.client.utils.ClientAuthenticationUtils.getAuthenticate(ClientAuthenticationUtils.java:92)
        at org.apache.arrow.driver.jdbc.client.ArrowFlightSqlClientHandler$Builder.build(ArrowFlightSqlClientHandler.java:968)
        at org.apache.arrow.driver.jdbc.ArrowFlightConnection.createNewClientHandler(ArrowFlightConnection.java:119)
        at org.apache.arrow.driver.jdbc.ArrowFlightConnection.createNewConnection(ArrowFlightConnection.java:89)
        at org.apache.arrow.driver.jdbc.ArrowFlightJdbcDriver.connect(ArrowFlightJdbcDriver.java:90)
        at org.apache.arrow.driver.jdbc.ArrowFlightJdbcDriver.connect(ArrowFlightJdbcDriver.java:46)
        at sqlline.DatabaseConnection.connect(DatabaseConnection.java:135)
        at sqlline.DatabaseConnection.getConnection(DatabaseConnection.java:192)
        at sqlline.Commands.connect(Commands.java:1487)
        at sqlline.Commands.connect(Commands.java:1361)
        at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:77)
        at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.base/java.lang.reflect.Method.invoke(Method.java:569)
        at sqlline.ReflectiveCommandHandler.execute(ReflectiveCommandHandler.java:44)
        at sqlline.SqlLine.dispatch(SqlLine.java:818)
        at sqlline.SqlLine.initArgs(SqlLine.java:447)
        at sqlline.SqlLine.begin(SqlLine.java:570)
        at io.gizmosql.sqlline.GizmoSQLLine.main(GizmoSQLLine.java:48)
Caused by: org.apache.arrow.driver.jdbc.shaded.io.netty.channel.AbstractChannel$AnnotatedConnectException: Connection refused: localhost/127.0.0.1:31337
Caused by: java.net.ConnectException: Connection refused
        at java.base/sun.nio.ch.Net.pollConnect(Native Method)
        at java.base/sun.nio.ch.Net.pollConnectNow(Net.java:672)
        at java.base/sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:946)
        at org.apache.arrow.driver.jdbc.shaded.io.netty.channel.socket.nio.NioSocketChannel.doFinishConnect(NioSocketChannel.java:336)
        at org.apache.arrow.driver.jdbc.shaded.io.netty.channel.nio.AbstractNioChannel$AbstractNioUnsafe.finishConnect(AbstractNioChannel.java:339)
        at org.apache.arrow.driver.jdbc.shaded.io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:784)
        at org.apache.arrow.driver.jdbc.shaded.io.netty.channel.nio.NioEventLoop.processSelectedKeysOptimized(NioEventLoop.java:732)
        at org.apache.arrow.driver.jdbc.shaded.io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:658)
        at org.apache.arrow.driver.jdbc.shaded.io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:562)
        at org.apache.arrow.driver.jdbc.shaded.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:998)
        at org.apache.arrow.driver.jdbc.shaded.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74)
        at org.apache.arrow.driver.jdbc.shaded.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
        at java.base/java.lang.Thread.run(Thread.java:840)

The only change I have made was adding 'sudo' to the docker run command since it wasn't working for me otherwise. Any ideas?

[prmoore77]

Hi @george-larionov - could you share the gizmosql container logs? I'll retry my test also to see if I get the same thing... thanks!

[prmoore77]

Hi @george-larionov - I think I figured it out. In my AWS EC2 provisioning scripts (attached) - I mount the recommended EBS volume to path: /nfs_data on the virtual machine. If that path isn't available, the GizmoSQL Docker container will crash (b/c we use that path to mount in the docker command - here: --mount type=bind,source=/nfs_data,target=/opt/gizmosql/data \).

Could you try with a volume mounted at that path?

Here are the scripts:
ec2_provisioning_scripts.zip

If you would like me to put the provisioning scripts in the codebase, just let me know. I didn't do so, b/c I didn't see that others had done so... Thanks for your help!

[prmoore77]

hi @george-larionov - were you able to test?

[george-larionov]

Hi @prmoore77, sorry for the late reply, it's been a busy week. I tried again after replacing /nfs_data with an existing path, but am still getting the same error, I don't think it is related to the mount path.

The expectation is that the benchmark.sh script can run on a fresh Ubuntu machine on AWS, so please include any specific mounting you need (although perhaps it would be simpler to just use the default mount paths). Take a look at the CedarDB or DuckDB scripts.

Additionally, the way the benchmarks are run is semi-automated using the run-benchmark.sh script, which fills in the cloud-init.sh script with the correct details and then runs it on the AWS instance, so take a look there for the exact way that things are run.

I'm also attaching the log.txt file, is this the one you meant?

Let me know if you have any questions, I will try to be more timely in my replies 😳

[rschu1ze]

Please see here for more details what warm / lukewarm and cold runs mean in the context of ClickBench.

Two comments related to that.

run.sh contains this:

# Execute all queries in one session (so authentication overhead is minimized)
echo "Running benchmark with $(wc -l < queries.sql) queries, ${TRIES} tries each..."

gizmosqlline \
  -u 'jdbc:arrow-flight-sql://localhost:31337?useEncryption=true&disableCertificateVerification=true' \
  -n clickbench \
  -p clickbench \
  -f "${TEMP_SQL_FILE}"

The script does not clear the OS page cache between query runs - which it should do to qualify as a "lukewarm run". If authentication is costly, then feel free to disable it entirely (the script seems to disable certificate validation already).

The other thing is that we are moving away from lukewarm runs. New submissions ideally do cold runs right from the start. Would it be possible to kill/start the GizmoSQL docker container between query runs?