OS/2 e-Zine! - the Rexx Files

the Rexx Files - by Dr. Dirk Terrell

In last month's column we wrote the code that creates the query that we need to send to the AltaVista web server. Now we need to write the code that talks to the server, sending the query and getting the response. Rexx itself does not contain the necessary functions for doing TCP/IP communications, but there is a freely available library that enables us to do just about anything we need to do involving socket programming.

Sockets are a way of virtualizing the underlying details of TCP/IP communications which are the heart of the Internet. Basically, a socket is a connection between two machines that enables them to send data back and forth to one another. With a library like RxSock, you create a socket connection and write to it like you would a file. The library handles all the gory details and saves you a lot of time.

If you've messed around with web servers or FTP daemons, you know that sockets are created on ports identified by numbers, and certain numbers are reserved for various functions. For example, web servers listen on port 80, telnet daemons on port 23, POP3 servers on port 110, and so on. (If you're curious, look at the services file in your \MPTN\ETC directory if you have a TCP/IP networking installed.) So, since we want to talk to the AltaVista web server, we will contact it by sending a message to port 80 via a socket connection.

The first thing we need to do is load the RxSock library. You can download it from the OS/2 Supersite. To load it, make sure that rxsock.dll is available in your LIBPATH somewhere and execute these two instructions:

   rc = RxFuncAdd("SockLoadFuncs","RxSock","SockLoadFuncs")
   rc = SockLoadFuncs()

There are five basic steps we need to perform:

Create a socket using the SockSocket() function
Connect the socket to the AltaVista server using SockConnect()
Send the query to the server using SockSend()
Receive the results from the server using SockRecv()
Close the socket using SockClose()

Step 1 - Create the Socket

Since we are identifying out server by its domain name "www.altavista.digital.com" we need to convert it to the dotted decimal IP address (204.74.103.37 for example). The RxSock library has a function for doing just this: SockGetHostByName(). By passing the domain name and a stemmed variable for the return values, we can get the IP address corresponding to that domain name:

   rc = SockGetHostByName("www.altavista.digital.com","host.!")

The stemmed variable passed as the second parameter uses the exclamation mark to avoid problems with stem tails having the same name as previously defined variables since the exclamation mark isn't commonly used to start variable names. You could use any valid character you like. Upon returning from the function call, the variable host!.addr will contain the IP address of the server.

To create our socket, we call SockSocket():

   socket  = SockSocket(domain,type,protocol)

The first parameter, domain is always "AF_INET" for the RxSock library. The second, type, can be "SOCK_STREAM", "SOCK_DGRAM", or "SOCK_RAW" and in our case we will use the "SOCK_STREAM" type. The third parameter, protocol, can be "IPPROTO_UDP", "IPPROTO_TCP", or "0" and we will use "0".

The SockSocket() function returns a value of -1 and sets the variable errno if there was a problem creating a socket. The SockGetHostByName() function returns a value of 0 if an error occurred. (See the RxSock documentation for the return values of various functions.) It is always a good idea to check the return value from a function call to make sure that things went smoothly and take appropriate action if not. So, our code to get the IP address and create a socket looks like this:

Say "Resolving hostname:" Site
rc = SockGetHostByName(Site,"host.!")
If (rc = 0) Then
      Do
         Say "Could not resolve hostname:" Site "Is the network down?"
         Return
      End

server = host.!addr;

/*  Open the socket */
Say "Opening socket..."
socket  = SockSocket("AF_INET","SOCK_STREAM",0)
If (socket = -1) Then
   Do
        Say "Error creating socket:" errno
        Return
   End

Step 2 - Connect the Socket to the Remote Server

Now that we have a socket, we need to connect it to the remote server with the SockConnect() function. The calling form is:

   rc = SockConnect(socket,address)

where socket is the socket we just created and address is a stem variable containing the address and port to which we will connect. The address stem must have a .port tail giving the port, a .addr tail containing the IP address of the server, and a .family tail set to "AF_INET". Once we have the address tail properly constructed, we can connect the socket, again looking at the return value from SockConnect() to see if any problems were encountered. (This function returns a -1 if an error occurred.) Thus, our connection code looks like this:

/* Connect the socket */
Say "Connecting to socket..."
server.!family = "AF_INET"
server.!port   = port
server.!addr   = server

rc = SockConnect(socket,"server.!")
If (rc = -1) then
   Do
        Say "Error on connecting socket:" errno
        Call CloseSock
        Return
   End

Notice that we call CloseSock to close the socket if an error occurs. Sockets do consume resources, so you need to ensure that you close them when you are finished with them. As you might guess, the SockClose() function performs this duty. Our CloseSock routine makes sure we close the socket properly:

CloseSock: 
/* Close the socket */

rc = SockSoClose(socket)
If (rc = -1) Then
   Do
        Say "Error closing socket:" errno)
   End
Return

Step 3 - Sending Our Query to the Search Engine

Having successfully connected to the server, we can now send it the query that we created in the last two articles. The SockSend() function sends data to the remote server. The calling form is:

   rc = SockSend(socket,data)

where socket is our socket and data is the data that we want to send to the server, namely our search query. Again, this function returns a -1 if there was a problem transmitting the data such as your dialup connection's dropping. To send our search query to AltaVista, we use this code:

/* Send the data to the remote server */
Say "Sending data to remote server..."
rc = SockSend(socket,Query)

If (rc = -1) Then
   Do
        tmp="Error sending data to server:" errno
        Call CloseSock
        Return
   End

Steps 4 and 5 - Receiving the Response from the Server and Closing the Socket

Once we send the data to the server, we wait for it to send a response to us. To capture the response we use the SockRecv() function whose calling form is:

   rc = SockRecv(socket,var,len)

where socket, once again, is our socket, var is a variable that we will use to store the data, and len is the maximum number or bytes to accept at one time. Since we don't know ahead of time how many bytes the server will return, we will call SockRecv() multiple times until all of the data have been read. The function returns a number greater than 0 as long as there are more data to be read. At each invocation, we will write out the results to a file for later analysis.

To give the user some indication that things are happening, we will print a # symbol to the screen every time the function is called. (This could, of course, be more sophisticated and print the symbol for every kilobyte of data received by keeping track of the number of bytes received during each read. Hint: Use the Length() function to get the number of bytes returned and then use the integer division operator %% to tell when a 1024-byte boundary has been crossed.) Once we have received all of the data, we will close the socket. The code to receive the data looks like this:

/* Receive the result from the server */
Say "Receiving data from remote server..."
File="results.html"
rc=SysFileDelete(File)
Do while SockRecv(socket, 'newdata', 1024) > 0
   Call Charout File,newdata
   Call Charout ,"#"
End
Call Charout crlf

If (rc = -1) then
   Do
        Say "Error receiving data from server:" errno
        Call CloseSock
   End
Else
   Call CloseSock

rc=Stream(File,"C","Close")

Conclusion

And there you have it: the code necessary to send a request to a web server and receive its response. What you do with it at this point is determined by your needs. You could parse the results and present a list to the user, or connect a socket to the web servers listed in the response and grab the documents that matched your search query. There are probably endless possibilities for this program. If you come up with any, let me know.

Dr. Dirk Terrell is an astronomer at the University of Florida specializing in interacting binary stars. His hobbies include cave diving, martial arts, painting and writing OS/2 software such as HTML Wizard.

[Our Sponsor: Indelible Blue - OS/2 software and hardware solutions to customers worldwide.]