Using sed and awk to manipulate files

In this post, I describe how I used two of the most useful of the Unix tools to manipulate a text file.

The file in question is the configuration file for IBM HTTP Server - httpd.conf, and I had a requirement to create a Bash script that would take the vanilla file, and enable SSL.

In essence, I wanted to go from this: -

...

#LoadModule ibm_ssl_module modules/mod_ibm_ssl.so
#Listen 443
#<VirtualHost *:443>
#SSLEnable
#</VirtualHost>
#KeyFile /opt/IBM/HTTPServer80/ihsserverkey.kdb
#SSLDisable
# End of example SSL configuration

...

to this: -

...

LoadModule ibm_ssl_module modules/mod_ibm_ssl.so
Listen 8443
<VirtualHost *:8443>
SSLEnable
</VirtualHost>
KeyFile /opt/IBM/HTTPServer80/ssl/WODMPCINT.kdb
#SSLDisable
# End of example SSL configuration
SSLCachePortFilename /opt/IBM/HTTPServer80/logsext/siddport
# The name of the socket to use for communication with the cgi daemon
ScriptSock logsext/cgisock
...

To achieve this, I created a script that makes use of two rather useful, but often misunderstood, Unix tools - sed and awk.

Firstly, I used sed ( or String Editor to give it its full name ) to search/replace instances of text within the file.

The syntax of the command is ( as I'm using it ) is: -

$ sed -i'' 's/<search>/<replace>/g' filename.ext

To be more specific, this is broken down as follows: -

$ sed <- The command

-i'' <- Perform an in-line edit, and do not create a backup file

's/ <- Start the search pattern

<search> <- The string for which to search

/ <- Separator

<replace> <- The string with which to replace

/g' <- End the search pattern AND search globally ( g )

filename.ext <- The file on which to perform the search/replace operation

This sequence worked for most of the lines: -

sed -i'' 's/Listen 8080/#Listen 8080/g' httpd.conf
sed -i'' 's/#LoadModule ibm_ssl_module/LoadModule ibm_ssl_module/g' httpd.conf
sed -i'' 's/#Listen 443/Listen 8443/g' httpd.conf
sed -i'' 's/#<VirtualHost \*:443>/<VirtualHost \*:8443>/g' httpd.conf
sed -i'' 's/#SSLEnable/SSLEnable/g' httpd.conf

but didn't work for the line that includes #</VirtualHost> because there were TWO instances of that line in the file, and I only wanted to uncomment the SECOND instance.

Therefore, after much Google'ing and much trial n' error, I discovered AWK ( supposedly this is the initials of it's creators - Aho, Weinberger and Kernighan ).

The specific command that I used was: -

awk '/#<\/VirtualHost>/{c++;if(c==2){sub("#<\/VirtualHost>","<\/VirtualHost>");c=0}}1' filename.ext > /tmp/foobar

which is broken down as follows: -

$ awk <- The command

'/#<\/VirtualHost> <- Searches for #</VirtualHost> - note the use of the \ escape character in front of the / character

{c++;if(c==2) <- Start a counter ( from 1 ), and wait until it reaches the required number - two ( 2nd instance )

{sub("#<\/VirtualHost>" <- The string for which to search

, <- Separator

"<\/VirtualHost>" <- The string with which to replace

);c=0}}1' <- Reset the counter and end the loop (?)

filename.exe <- The file on which to perform the search/replace operation

> /tmp/foobar <- The temporary output file

I'm not sure whether awk ( AWK ) has an in-line edit option, like sed's -i'' hence the use of a temporary file.

So here's the finished script: -

sed -i'' 's/Listen 8080/#Listen 8080/g' httpd.conf
sed -i'' 's/#LoadModule ibm_ssl_module/LoadModule ibm_ssl_module/g' httpd.conf
sed -i'' 's/#Listen 443/Listen 8443/g' httpd.conf
sed -i'' 's/#<VirtualHost \*:443>/<VirtualHost \*:8443>/g' httpd.conf
sed -i'' 's/#SSLEnable/SSLEnable/g' httpd.conf
awk '/#<\/VirtualHost>/{c++;if(c==2){sub("#<\/VirtualHost>","<\/VirtualHost>");c=0}}1' httpd.conf > /tmp/foobar

cp /tmp/foobar httpd.conf
sed -i'' 's/#KeyFile/KeyFile/g' httpd.conf
sed -i'' 's/ihsserverkey.kdb/ssl\key.kdb/g' httpd.conf
sed -i '863iSSLCachePortFilename /opt/IBM/HTTPServer80/logsext/siddport' httpd.conf
sed -i '864i# The name of the socket to use for communication with the cgi daemon' httpd.conf
sed -i '865iScriptSock logsext/cgisock' httpd.conf
echo "# WebSphere Plugin" >> httpd.conf
echo 'LoadModule was_ap22_module "/opt/IBM/HTTPPlugins80/bin/64bits/mod_was_ap22_http.so"' >> httpd.conf
echo 'WebSpherePluginConfig /opt/IBM/HTTPPlugins80/config/IHS/plugin-cfg.xml' >> httpd.conf

Simple :-)