HomeUsing VS Accessing Web Sites Using Desktop Applications
Accessing Web Sites Using Desktop Applications ByJohn Mueller 2004-02-21
Article Rating: / 4
Rate This Article:
Add This Article To:
Desktop and Web applications really can interact. Solve your mixed coding environment problems using the tips in this article.
Web applications don't always require a browser for interaction. Sometimes, a company will ask me to create a desktop application that interacts with a Web site, so they can reduce training and support costs, as well as the number of interfaces users must learn. Although the application still interacts with the Web site, the user isn't aware of the process.
In some cases, the request for a "desktop" Web application is a matter of integration. The desktop application provides a single point of access, or a central point of communication with multiple sources. Some clients ask for desktop
interfaces for Web applications because browsers don't necessarily provide the best control over user interaction.
ADVERTISEMENT
No matter what your need is, you can effectively interact with a Web site using a desktop application, without the user knowing that a Web site is in use. I'll show you how.
A Simple Scenario
The .NET Framework provides several classes for working with online sources. For this simple example, you'll use one class for requesting information and a second class to receive the response. These classes all derive from System.Net.WebRequest and System.Net.WebResponse. The current version of the .NET Framework uses HttpWebRequest for Web page requests and FileWebRequest for online file requests, with similar classes for responses. To work with Web pages, you'll use the HttpWebRequest and HttpWebResponse classes.
A good way to begin with desktop interaction with a Web site is to request a page. Then you use a technique called screen scraping to retrieve the information from the Web page. You don't have to do anything odd to make the interaction work, but using this method lets the code search the Web page for a label that contains some information; it could be as simple as a response to a help question.
<html>
<head>
<meta http-equiv="Content-Type"
content="text/html;
charset=windows-1252">
<title>Simple Help Page</title>
</head>
<body>
<h1>Simple Page</h1>
<label id="HelpText"
name="HelpText">This is some help text</label></p>
</body>
</html>
The code to request the Web page and read its content is relatively straightforward. Listing 1 shows a typical example of getting a Web page.
Listing 1: Basic Structure of a Web Interaction Program
private void btnTest_Click(object sender, System.EventArgs e)
{
HttpWebRequest Req; // The Request.
HttpWebResponse Resp; // The response.
Stream Data; // Response data stream.
StreamReader DatRead; // Obtains the data stream.
String Output; // The response as a string.
Int32 Posit1; // Beginning position within the string.
Int32 Posit2; // End position within the string.
// Create the initial request.
Req = (HttpWebRequest)WebRequest.Create(txtURL.Text);
try
{
// Get the response.
Resp = (HttpWebResponse)Req.GetResponse();
// Read the data from the response.
Data = Resp.GetResponseStream();
DatRead = new StreamReader(Data);
// Get the data as a string.
Output = DatRead.ReadToEnd();
// Always close the data stream.
DatRead.Close();
}
catch (WebException WE)
{
// Display an error message and exit.
MessageBox.Show(WE.Message);
return;
}
// Look for the beginning of the label entry.
Posit1 = Output.IndexOf("HelpText");
Posit1 = Output.IndexOf(">", Posit1)
Posit1++;
// Look for the ending of the label entry.
Posit2 = Output.IndexOf("<", Posit1);
// Display the resulting string on screen.
MessageBox.Show(Output.Substring(Posit1, Posit2 - Posit1));
}
The code begins by creating an HttpWebRequest object, Req, which represents the connection to the server. (It transmits the request for a specific page to the server, so you need to recreate it whenever you want to change pages.) Once the server receives the request, it prepares to send a responseÑno matter what that response might be.
After the code sends the request, it creates an HttpWebResponse object, Resp. Notice that this object is created using the Req.GetResponse() method, because Req still represents the connection with the server. Resp is now the response connection to the server. The code uses a data stream to receive the response from the server, and then a StreamReader to create a local copy of the data in memory.
At this point, you can begin working with the data. The first step is to place the page in a string, Output. Output contains the whole page, just as you would see it using the View | Source command in Internet Explorer.
Now that there's a local copy of your data, make sure you close the StreamReader object, DatRead. All of these reading steps occur within a try...catch block. The reason for this precaution is that you want to catch any server responses that equate to an error. In fact, you can display the server response as part of the error message, if you want.
The screen scraping portion of the code consists of locating the information you need. This is a situation where good coding practice comes into play. Using labels and other recommended constructs makes it easier to interact with the page. In this case, all you need to look for is the unique label name, HelpText. Because this name appears within the label tag, the code looks for the end of the tag and uses that as the starting point for the data. The ending point is easy to find because the label tag ends with a </label> entry. All the code needs to do is locate the opening angle bracket for this tag.
Using the GET Method
Some of the Web pages you design will require one or more types of interaction. Generally, a user fills out a form, clicks a button, and sends it off to you. One technique used to transmit the content is called the GET method. That's the method where
you see an URL, followed by a question mark, followed by some data entries, like this:
This page has one variable, Msg, which equals Hello World.
The desktop application user can't fill out a form as a Web page user would. Sure, you can place the required data on a form, and the user can click a button, but the form doesn't react the same as a browser form. Fortunately, you can modify the previous example to work with this new scenario. All you need to do is modify the URL you create using a technique like this:
// Create the output string.
Output = "?Msg=" + txtData.Text;
// Create the initial request.
Req = (HttpWebRequest)WebRequest.Create(txtURL.Text + Output);
The only real change is that the code adds the variables to the
URL. The GET method makes sending variables to the page simple.
Using the POST Method
Not every Web page uses the GET method of accepting input. Many developers use the POST method to accept input, so the user doesnÕt actually see the form variables as part of the URL. In addition, the POST method can overcome limitations within some browsers regarding content length for a GET method call.
Unfortunately, the technique for working with a POST method page is a little more involved. Here's a typical example of the modifications required for the POST method.
// Create the initial request.
Req = (HttpWebRequest)WebRequest.Create(txtURL.Text);
// Create the output string.
Output = "Msg=" + txtData.Text;
// Prepare the data for output.
Encode = new ASCIIEncoding();
PostDat = Encode.GetBytes(Output);
// Set the Request up to post information to the server.
Req.ContentLength = PostDat.Length;
Req.ContentType = "application/x-www-form-urlencoded";
Req.Method = "POST";
// Output the data to the server.
Data = Req.GetRequestStream();
Data.Write(PostDat, 0, PostDat.Length);
// Close the output stream.
Data.Close();
The request itself doesnÕt contain any data because the post method writes the data separately. The posting method begins by creating the message variables. (Don't forget to add an ampersand between each variable set.)
Once the code creates a string with all the variables, it encodes the string and outputs it as a Byte array. This Byte array contains the final form of the data, but before the code can transmit the data, it must adjust the request. The example shows the minimal changes you can perform and still send the data. The ContentLength, ContentType, and Method properties must contain values that reflect the new status of the request.
Now it's time to send the data. This time, the code creates a request stream and outputs the variables to the server. Make absolutely certain that you close the request stream before you create a response stream, or the application will fail.
A Few Words about Security
All the procedures in this article work, whether you access a standard or a secure Web page. However, secure Web pages require a few more additions.
To access a Web page that relies only on a password for access, you can create a credential cache to the server. Listing 2 shows a function that creates a request and adds credentials to it.
Listing 2: Adding Credentials to a Request
private HttpWebRequest CreateCache()
{
HttpWebRequest Req; // The Request.
NetworkCredential OneCred; // One credential.
CredentialCache CredCache; // All of the credentials.
// Create the initial request.
Req = (HttpWebRequest)WebRequest.Create(txtURL.Text);
// Create a single credential.
OneCred = new NetworkCredential("UserName", "UserPassword");
// Create the cache based on the Web page URL and authentication
// type.
CredCache = new CredentialCache();
CredCache.Add(new Uri(txtURL.Text), "Basic", OneCred);
// Add the cache to the request.
Req.Credentials = CredCache;
// Return the request.
return Req;
}
However, if your connection relies on SSL (the HTTPS protocol), you must also change the way the application reacts to a certificate from the server. There are two steps to this process. First, you set the ServicePointManager to accept certificates. This means setting it to point to a special class that defines this policy. HereÕs a typical example:
// Add certificate support.
ServicePointManager.CertificatePolicy =
new AcceptAllCertificatesPolicy();
Of course, you also have to create the policy. That's relatively easy. All you need to do is create a class that inherits from the ICertificatePolicy interface. HereÕs a sample of that class:
class AcceptAllCertificatesPolicy : ICertificatePolicy
{
#region ICertificatePolicy Members
public bool CheckValidationResult(
ServicePoint srvPoint,
System.Security.Cryptography.X509Certificates.X509Certificate certificate,
WebRequest request, int certificateProblem)
{
// TODO: Add AcceptAllCertificatesPolicy.CheckValidationResult
// implementation
return true;
}
#endregion
}
The CheckValidationResult() method receives information about the certificate. You need to add code that determines whether the certificate is valid. If it is, then you return true to the caller; otherwise you return false.
As you can see, interacting with Web pages using a desktop application is more involved than using a browser. However, it's possible to create a completely seamless experience for desktop users with the code presented in this article. The process is always the same: a request and then a response. The hard part is defining what you need to provide as input for the request and what to expect as output from the server. In most cases, a little experimentation goes a long way toward solving both problems.
John Mueller is a freelance author and technical editor. He has writing in his blood, having produced 62 books and over 300 articles to date. The topics range from networking to artificial intelligence and from database management to heads down programming. His most recent book is Mining Amazon Web Services (Sybex, ISBN:0782143075).
/ 4 
