[Go-essp-tech] Apache Http Proxy Configuration for esg-gateway.jpl.nasa.gov

Fri Mar 18 14:51:02 MDT 2011

Hi Paul,

Thank you for another excellent reply.  I was actually working with 
mod_proxy_ajp this morning and was also able to get the client 
certificate passed though to tomcat.

Here is a thread that was posted on Server Fault:
http://serverfault.com/questions/248797/apache-not-forwarding-client-x509-certificate-to-tomcat-via-mod-proxy

That is cleaver cleaning out the ssl headers.  Is header forgery a 
common problem?  It would be nice if mod_proxy_ajp did that automajically.

Do you know if setting the remote port is as easy as setting the 
REMOTE_PORT header using the RequestHeader set command that you're using 
for the SSL attributes?  Please see the bottom section of "AJP as a 
Solution" on the following link for more information:
http://tomcat.apache.org/connectors-doc/generic_howto/proxy.html

I'm really glad that a.) both environments showed the same behavior with 
certificates when using mod_proxy and b.) that we all came up with the 
relatively the same solution individually.  I am quite confident at this 
point, that using mod_proxy_ajp via apache is the correct solution for 
our safari issues.

Thank you again for your time and efforts Paul and Luca!  Very much 
appreciated.

Regards,

Nathan H.

On 3/18/2011 2:28 PM, Paul Zimdars wrote:
> Hi Nathan,
>
> I believe I have a fix that will allow the certificate to pass from
> Apache to Tomcat (and not break Safari). Here are the steps:
>
> 1. Rebuild apache and add the "--enable-headers" flag:
>
> ./configure --with-ssl=/usr/local/openssl-0.9.8k
> --prefix=/usr/local/apache-2.2.15 --enable-proxy --enable-ssl
> --enable-cgi --enable-suexec --enable-rewrite --enable-so
> --enable-proxy-connect --enable-proxy-http --enable-proxy-ajp --
> enable-ldap --enable-dav --enable-vhost-alias --enable-http
> --enable-unique-id --enable-headers
>
> 2. Configure the tomcat connector to use AJP instead of HTTP
> (protocol="AJP/1.3").
>
> 3. Edit your httpd to reflect the AJP change and add the header request
> configuration. Here is my new configuration:
>
> <VirtualHost *:443>
> ServerAlias esg-gateway
> SSLEngine on
> SSLProxyEngine on
> SSLProtocol all
> ProxyRequests Off
> ProxyPreserveHost On
> ServerName esg-gateway.jpl.nasa.gov
> SSLCipherSuite HIGH:MEDIUM
> SSLCertificateFile /etc/httpd/conf/certs/esg-gw.crt
> SSLCertificateKeyFile /etc/httpd/conf/certs/esg-gw.key
> SSLVerifyClient optional_no_ca
> SSLOptions +ExportCertData
> <Proxy *>
> AddDefaultCharset Off
> Order deny,allow
> Allow from all
> </Proxy>
> ServerAdmin sa at list.jpl.nasa.gov
>
> # initialize the special headers to a blank value to avoid http header
> forgeries
>
> RequestHeader set SSL_CLIENT_S_DN ""
> RequestHeader set SSL_CLIENT_I_DN ""
> RequestHeader set SSL_SERVER_S_DN_OU ""
> RequestHeader set SSL_CLIENT_VERIFY ""
>
> <Location />
>
> # add all the SSL_* you need in the internal web application
>
> RequestHeader set SSL_CLIENT_S_DN "%{SSL_CLIENT_S_DN}s"
> RequestHeader set SSL_CLIENT_I_DN "%{SSL_CLIENT_I_DN}s"
> RequestHeader set SSL_SERVER_S_DN_OU "%{SSL_SERVER_S_DN_OU}s"
> RequestHeader set SSL_CLIENT_VERIFY "%{SSL_CLIENT_VERIFY}s"
>
> ProxyPass ajp://localhost:9443/
> ProxyPassReverse ajp://localhost:9443/
> </Location>
> </VirtualHost>
>
> That should do the trick..we tested it our end and the site still works
> with Safari and Luca was able to use the ESG publisher.
>
> Thanks,
> Paul Zimdars
> 626-824-1393
>
> On 3/18/2011 11:46 AM, Cinquini, Luca (3880) wrote:
>> Hi Nathan,
>> I experimented again with publishing to the JPL gateway, and so far
>> our findings (plus what I am starting to recall) confirms your
>> findings... With the current Apache-Tomcat configuration, the user
>> certificate does NOT seem to be transmitted from Apache to Tomcat.
>>
>> So, if I use the following configuration in the ESG publisher:
>>
>> hessian_service_port = 443
>> hessian_service_url =
>> https://esg-gateway.jpl.nasa.gov/remote/secure/client-cert/hessian/publishingService
>>
>>
>> I get the following error in the publisher, which means the Gateway
>> does not have a certificate and is trying to redirect the request to
>> the login page, I believe:
>>
>> File
>> "/usr/local/cdat/lib/python2.6/site-packages/esgcet-2.7.4-py2.6.egg/esgcet/publish/publish.py",
>> line 308, in publishDatasetList
>> dset, statusId, state, evname, status = publishDataset(datasetName,
>> parentIdent, service, threddsRootURL, session)
>> File
>> "/usr/local/cdat/lib/python2.6/site-packages/esgcet-2.7.4-py2.6.egg/esgcet/publish/publish.py",
>> line 118, in publishDataset
>> statusId = service.createDataset(parentId, threddsURL, -1, "Published")
>> File
>> "/usr/local/cdat/lib/python2.6/site-packages/esgcet-2.7.4-py2.6.egg/esgcet/publish/hessianlib.py",
>> line 426, in __call__
>> return self._invoker(self._method, args)
>> File
>> "/usr/local/cdat/lib/python2.6/site-packages/esgcet-2.7.4-py2.6.egg/esgcet/publish/hessianlib.py",
>> line 537, in __invoke
>> raise ProtocolError(self._url, errcode, errmsg, headers)
>> esgcet.publish.hessianlib.ProtocolError:<ProtocolError for
>> https://esg-gateway.jpl.nasa.gov/remote/secure/client-cert/hessian/publishingService:
>> 302 Move
>>
>> But if I use the following configuration, which queries port 9443 on
>> the Tomcat server directly:
>>
>> # Publish to NCAR Gateway
>> hessian_service_port = 9443
>> hessian_service_url =
>> https://esg-gateway.jpl.nasa.gov/remote/secure/client-cert/hessian/publishingService
>>
>>
>> everything works fine. So, now that I recall, I did notice this
>> problem several months ago and, after experimentation, figure out that
>> you should publish to Tomcat directly.
>>
>> I think Paul might be trying to see if he can change the Apache con to
>> allow publishing to it directly, but in any case the above setup
>> should allow you to publish data, and to avoid the Safari truststore
>> problem.
>>
>> thanks, Luca
>>
>> On Mar 17, 2011, at 11:41 AM, Nathan Hook wrote:
>>
>>> Hi Luca,
>>>
>>> Thank you for the reply.
>>>
>>> When you test publishing could you verify if you're publishing to
>>> either: esg-gateway.jpl.nasa.gov or jpl-esg.jpl.nasa.gov?
>>>
>>> I've tried verifying that our setup is the same as the one setup at JPL,
>>> but there does seem to be a communication problem between apache and
>>> tocmat.
>>>
>>> There are some software version differences, but I'm not sure those by
>>> themselves could be causing the problems.
>>>
>>> Here are our software versions:
>>> Apache/2.2.3
>>> Tomcat/6.0.29
>>> Java/6.0_23
>>>
>>> The could be issues with SSL Renegotiation...
>>>
>>> Thanks again Luca.
>>>
>>> Regards,
>>>
>>> Nathan H.
>>>
>>>
>>>
>>>
>>> On 3/17/2011 11:28 AM, Cinquini, Luca (3880) wrote:
>>>> Hi Nathan,
>>>> this is probably not going to be very useful, but just to give you
>>>> an update...
>>>>
>>>> In the past, I have been able to publish to the JPL Gateway via the
>>>> standard ESG publisher, which means that the proxy certificate was
>>>> indeed getting to the Tomcat server. Yesterday I started trying to
>>>> reproduce the steps to give you some debug information, but I
>>>> realized I had to upgrade our datanode first. So that's what I have
>>>> been doing, and as soon as I am done I can report back on the
>>>> results. It might take till tomorrow since I have a few meetings
>>>> this pm.
>>>>
>>>> Sorry I can't be more helpful at this time.
>>>>
>>>> thanks, Luca
>>>>
>>>> On Mar 16, 2011, at 4:52 PM, Nathan Hook wrote:
>>>>
>>>>> Hi Paul,
>>>>>
>>>>> Thank you for the very complete explanation of your environment's
>>>>> setup.
>>>>> It has been very helpful so far.
>>>>>
>>>>> I do seem to be having difficultly getting a client's x509 certificate
>>>>> to the tomcat server at this point. I am testing this functionality by
>>>>> importing a p12 certificate into my browser and then visiting a page
>>>>> that uses ssl.
>>>>>
>>>>> Do you happen to know of a reasonable way to debug what might be going
>>>>> wrong?
>>>>>
>>>>> After snooping both the Apache ssl logs and the ssl debug output of
>>>>> Tomcat I'm a little nervous that the client's x509 certificate is not
>>>>> being sent from Apache to Tomcat.
>>>>>
>>>>> Any reasonable suggestions on either what might be wrong or how to
>>>>> debug
>>>>> the situation would be greatly appreciated.
>>>>>
>>>>> Thank you for your time Paul,
>>>>>
>>>>> Nathan H.
>>>>>
>>>>>
>>>>> On 3/16/2011 4:16 AM, Zimdars, Paul A (3880-Affiliate) wrote:
>>>>>> Hi Nathan,
>>>>>>
>>>>>> Here is an overview of our site deployment:
>>>>>>
>>>>>> CentOS 5.5 x86_64
>>>>>> Apache HTTPD 2.2.15
>>>>>> OpenSSL 0.9.8k (required for SNI)
>>>>>> Apache Tomcat 6.0.20 (Deployed by Luca)
>>>>>> JDK 1.6.0_13 (Deployed by Luca)
>>>>>>
>>>>>> We first installed OpenSSL 0.9.8k in order to use SNI (Server Name
>>>>>> Indication) which allows multiple SSL virtual hosts to share the
>>>>>> same IP address while providing unique SSL certificates for each
>>>>>> host. I recommend that you read more about it at the following site:
>>>>>>
>>>>>> http://wiki.apache.org/httpd/NameBasedSSLVHostsWithSNI
>>>>>>
>>>>>> We have multiple SSL virtual hosts using the same IP so SNI was
>>>>>> required for our site. We did a standard OpenSSL install with no
>>>>>> special flags (deployed to /usr/local/openssl-0.9.8k).
>>>>>>
>>>>>> The next step involved installing Apache 2.2.15 using the OpenSSL
>>>>>> deployment mentioned above + the following flags:
>>>>>>
>>>>>> --with-ssl=/usr/local/openssl-0.9.8k
>>>>>> --prefix=/usr/local/apache-2.2.15 --enable-proxy --enable-ssl
>>>>>> --enable-cgi --enable-suexec --enable-rewrite --enable-so
>>>>>> --enable-proxy-connect --enable-proxy-http --enable-proxy-ajp
>>>>>> --enable-ldap --enable-dav --enable-vhost-alias --enable-http
>>>>>> --enable-unique-id
>>>>>>
>>>>>> We then configured the VHOST for esg-gateway (80,443). We tried a
>>>>>> couple of different configurations but found that the
>>>>>> configuration listed below worked well (we also made esg-gateway
>>>>>> the first vhost found in httpd.conf since we had a couple of minor
>>>>>> issues with wget if it wasn't first (wget doesn't know about SNI)):
>>>>>>
>>>>>> <VirtualHost *:80>
>>>>>> ServerName esg-gateway.jpl.nasa.gov
>>>>>> ServerAlias esg-gateway
>>>>>> ProxyRequests Off
>>>>>> ProxyPreserveHost On
>>>>>> <Proxy *>
>>>>>> AddDefaultCharset Off
>>>>>> Order deny,allow
>>>>>> Allow from all
>>>>>> </Proxy>
>>>>>> ProxyPass / http://jpl-esg.jpl.nasa.gov:9080/
>>>>>> ProxyPassReverse / http://jpl-esg.jpl.nasa.gov:9080/
>>>>>> ServerAdmin sa at list.jpl.nasa.gov
>>>>>> </VirtualHost>
>>>>>>
>>>>>> <VirtualHost *:443>
>>>>>> ServerAlias esg-gateway
>>>>>> SSLEngine on
>>>>>> SSLProxyEngine on
>>>>>> SSLProtocol all
>>>>>> ProxyRequests Off
>>>>>> ProxyPreserveHost On
>>>>>> ServerName esg-gateway.jpl.nasa.gov
>>>>>> SSLCipherSuite HIGH:MEDIUM
>>>>>> SSLCertificateFile /etc/httpd/conf/certs/esg-gw.crt
>>>>>> SSLCertificateKeyFile /etc/httpd/conf/certs/esg-gw.key
>>>>>> SSLVerifyClient optional_no_ca
>>>>>> SSLOptions +ExportCertData
>>>>>> <Proxy *>
>>>>>> AddDefaultCharset Off
>>>>>> Order deny,allow
>>>>>> Allow from all
>>>>>> </Proxy>
>>>>>> ProxyPass / https://esg-gateway.jpl.nasa.gov:9443/
>>>>>> ProxyPassReverse / https://esg-gateway.jpl.nasa.gov:9443/
>>>>>> ServerAdmin sa at list.jpl.nasa.gov
>>>>>> </VirtualHost>
>>>>>>
>>>>>> The tomcat installation was fairly standard so I won't provide
>>>>>> details on it. The keystore was purchased from RapidSSL.com and I
>>>>>> roughly followed the following site to create and install the
>>>>>> keystore:
>>>>>>
>>>>>> http://www.agentbob.info/agentbob/79-AB.html
>>>>>>
>>>>>> Which provided me with our keystore that we added to Tomcat:
>>>>>>
>>>>>> Keystore type: JKS
>>>>>> Keystore provider: SUN
>>>>>>
>>>>>> Your keystore contains 1 entry
>>>>>>
>>>>>> 1, Mar 16, 2011, PrivateKeyEntry,
>>>>>> Certificate fingerprint (MD5):
>>>>>> 82:03:51:77:CC:4B:F2:8E:44:92:21:C5:BA:FE:39:92
>>>>>>
>>>>>> I do notice that if I use safari (5.0.4) in Windows I have issues
>>>>>> accessing esg-gateway over SSL (works fine on my Mac). I also
>>>>>> tested using Safari on my Mac (OSX 10.6.6/5.0.4) to directly
>>>>>> access our Tomcat installation (bypassing apache) and it fails on
>>>>>> the SSL connection.
>>>>>>
>>>>>> Please let me know if you have any further questions or no more
>>>>>> information.
>>>>>>
>>>>>> Thanks,
>>>>>> Paul
>>>>>> 626-824-1393
>>>>>> ________________________________________
>>>>>> From: Nathan Hook [nhook at ucar.edu]
>>>>>> Sent: Tuesday, March 15, 2011 11:13 AM
>>>>>> To: Zimdars, Paul A (3880-Affiliate)
>>>>>> Cc: go-essp-tech at ucar.edu
>>>>>> Subject: Apache Http Proxy Configuration for esg-gateway.jpl.nasa.gov
>>>>>>
>>>>>> Good Day Paul,
>>>>>>
>>>>>> My name is Nathan Hook with the National Center for Atmospheric
>>>>>> Research
>>>>>> (NCAR) and I have been working with a colleague of yours Luca
>>>>>> Cinquini
>>>>>> on a SSL Handshake problem with a product of ours.
>>>>>>
>>>>>> I am hoping that if you have the time available you would be able to
>>>>>> help me out a bit.
>>>>>>
>>>>>> Currently a collaboration that I'm working on (with Luca and
>>>>>> others) is
>>>>>> having difficulties with a ssl handshake being too large for the
>>>>>> Safari
>>>>>> web browser buffer and therefore causing Safari users not to
>>>>>> access our
>>>>>> web application via ssl. Which, is a problem.
>>>>>>
>>>>>> Luca has mentioned that the esg-gateway.jpl.nasa.gov site is proxied
>>>>>> behind an Apache Http server and it seems as though that
>>>>>> configuration
>>>>>> does not cause the same buffer problem for Safari that a stand alone
>>>>>> Apache Tomcat server does.
>>>>>>
>>>>>> Would you be willing to share with us your configuration settings for
>>>>>> both apache and tomcat so we can attempt to duplicate your
>>>>>> environment
>>>>>> here on our test servers and see exactly what is happening during
>>>>>> a ssl
>>>>>> request?
>>>>>>
>>>>>> Thank you for your time Paul.
>>>>>>
>>>>>> Regards,
>>>>>>
>>>>>> Nathan Hook
>>>>>>
>>>>> _______________________________________________
>>>>> GO-ESSP-TECH mailing list
>>>>> GO-ESSP-TECH at ucar.edu
>>>>> http://mailman.ucar.edu/mailman/listinfo/go-essp-tech
>>> _______________________________________________
>>> GO-ESSP-TECH mailing list
>>> GO-ESSP-TECH at ucar.edu
>>> http://mailman.ucar.edu/mailman/listinfo/go-essp-tech
>