Skip to content

GH-3482: parquet-hadoop tests to work behind a web proxy#3483

Open
alexeyroytman wants to merge 4 commits intoapache:masterfrom
alexeyroytman:master
Open

GH-3482: parquet-hadoop tests to work behind a web proxy#3483
alexeyroytman wants to merge 4 commits intoapache:masterfrom
alexeyroytman:master

Conversation

@alexeyroytman
Copy link
Copy Markdown

Rationale for this change

Allow parquet-hadoop tests to work behind a web proxy. Otherwise their failures stop some other jar files from being build.

What changes are included in this PR?

Analysis of system properties https.proxyHost and https.proxyPort, and if set, use a Proxy for OkHttpClient creation.

Are these changes tested?

The ./mvn clean build now passes when I'm behind a web proxy.

Are there any user-facing changes?

No.

Copy link
Copy Markdown
Contributor

@Fokko Fokko left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @alexeyroytman This seems reasonable to add

@alexeyroytman
Copy link
Copy Markdown
Author

alexeyroytman commented Apr 16, 2026

It looks like that on CI Hadoop the JVM properties https.proxyHost and https.proxyPort are set (I don't see their values, but I suppose they are set), and this specific proxy does not allow good gzip/snappy work.
Strangely, for JDK 11 tests passed, but for JDK 17 they failed...

I'll try to work on my environment with OpenJDK 17, and if it passes, I'm planning to change the code to use different JVM properties, e.g. parquet.https.proxyHost and parquet.https.proxyPort. Thus they won't be occasionally defined for CI. And occasionally not working for certain compression method.

@alexeyroytman
Copy link
Copy Markdown
Author

BTW, how can I find out which CI job has JVM properties https.proxyHost and https.proxyPort set ?

@alexeyroytman
Copy link
Copy Markdown
Author

Or shall we rename the properties to something referring to github.com ?.. I mean, it's parquet, hadoop, tests/junit, and for https://github.com ...

@alexeyroytman
Copy link
Copy Markdown
Author

I believe it shall pass CI now.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants