How can I use openssl to get results from HTTP GET requests?

conorgriffin asked:

I need to use openssl to perform some HTTP GET requests in a shell script. The line I’m using to do this right now is shown below. This is parsing the content of an XML response of the following formats.



echo -e "GET /test HTTP/1.1\r\nHost:$(hostname)\r\n\r\n" | openssl 2>&1 s_client -quiet -connect server-url:443 | grep -o -P --color '(?<=Result\>).*(?=\</Result)'

This works and returns the string ‘success’ or ‘failure’ accordingly. The problem I’m facing is that the openssl command does not terminate after doing the GET request but instead sits there waiting on more input. I believe this is due to the implicit -ign_eof which prevents automatic termination caused by the -quiet option. I’ve tried using the -no_ign_eof option but that causes the openssl command to terminate before the GET request has received a response so I can’t get the content of the response if I use that.

How can I modify this command so I can pass the GET request through stdin (required as I want to put this in a loop) but have the openssl command terminate after each request?

My answer:

What you really should be doing is using a tool designed for fetching Web resources, such as curl, wget, or libwww-perl’s GET command. If nothing is available, you should have your system administrator install something appropriate.

With that out of the way…

The openssl command does not terminate because the web server didn’t close the connection.

Remember that by default HTTP keeps connections open after each request as a performance optimization. Once one request finishes, another request can be sent over the same connection, rather than closing and reopening a new connection.

If you want to instruct the server to close the connection instead, you can send the Connection: close HTTP header.

View the full question and any other answers on Server Fault.

Creative Commons License
This work is licensed under a Creative Commons Attribution-ShareAlike 3.0 Unported License.