< VB .NET Http Post Request Program Example | Main | C++ Asynchronous Http Get Request Program Example >
Chapter 10 Part 8:
HTTP with .NET
What do we have in this chapter 10 Part 8?
-
Web Proxies
-
WebRequest and WebResponse
-
Cookies
-
Asynchronous HTTP Model
Web Proxies
Corporate environments often incorporate a firewall that requires a proxy server to be specified for Web traffic to travel beyond the corporate network. If a request is made for a resource outside the local proxied network, the request will fail and a WebException is thrown. For Web requests to traverse the proxy, the Proxy property of the HttpWebRequest class must be set. To set the proxy, an instance of the WebProxy class is created with the string name of the proxy server. The following code illustrates creating a HttpWebRequest and setting the Proxy property. Notice that the property needs to be set before invoking the request with the GetResponse() method.
C#
WebProxy proxyServer = new WebProxy("http://corpproxy/"); HttpWebRequest httpRequest = (HttpWebRequest) WebRequest.Create("http://www.winisp.net/goodrich"); httpRequest.Proxy = proxyServer; HttpWebResponse httpResponse = (HttpWebResponse) httpRequest.GetResponse();
Visual Basic .NET
Dim proxyServer As WebProxy = New WebProxy("http://corpproxy:80") Dim httpRequest As HttpWebRequest = WebRequest.Create("http://www.winisp.net/goodrich") httpRequest.Proxy = proxyServer HttpWebResponse httpResponse = httpRequest.GetResponse()
In the preceding example, a specific proxy server is set for the request by specifying the proxy server name in the WebProxy constructor. If Microsoft Internet Explorer is configured with a static proxy server, the static WebProxy method GetDefaultProxy() will return the currently configured proxy server as a WebProxy object, as shown by the following statement:
WebProxy proxyServer = WebProxy.GetDefaultProxy();
If the proxy server requires authentication, the necessary credentials to access the server must be specified in the Credentials property of the WebProxy class. Creating and assigning network credentials is covered in detail later on in this chapter. The newer version of the .NET Framework is likely to add automatic proxy discovery, that is, support Web Proxy Auto Discovery (WPAD). |
|
In addition to the ability to set the proxy server information on each request, the .NET Framework maintains a global proxy setting for use as the default proxy for all Web requests if a proxy setting is not explicitly provided. This is done through the GlobalProxySelection class. The property of interest is Select, which is assigned a WebProxy object that will be the default proxy server for subsequent requests when a proxy server is not explicitly assigned. The GlobalProxySelection.GetEmptyWebProxy() method is of interest because it can be assigned to GlobalProxySelection.Select to turn off all use of a proxy server across the entire application.
Finally, a default proxy setting for System.Net can be specified in the machine.config file located in the .NET Framework runtime installation directory usually found under <drive>:\<windir>\Microsoft.Net\Framework\<version>\config. The default setting should appear as follows:
<system.net>
<defaultProxy>
<proxy usesystemdefault="true"/>
</defaultProxy>
</system.net>
The following screenshot is for version 1.x (C:\WINDOWS\Microsoft.NET\Framework\v1.1.4322\CONFIG). For version 2.x we need to add it manually and for version 3.x, seemed machine.config cannot be found under the previous path.
This setting indicates that System.Net will use the computer’s proxy settings, as specified in the Internet Options control panel applet. The system default settings for proxy servers will generally suffice and applications do not have to set a specific proxy on the request.
WebRequest and WebResponse
The WebRequest and WebResponse classes offer protocol agnostic abstract interfaces to request-response communication. Currently, the .NET Framework offers handlers for file and HTTP transfers, and it is possible for other protocols to be plugged into the same harness by inheriting from the WebRequest and WebResponse classes.
We’ve seen how the WebRequest class plays a role in issuing an HTTP request. Earlier, when we created an HTTP request, we called the WebRequest.Create() method with the URI of the resource being requested. Because the URI begins with the protocol http, the WebRequest class is able to determine that the HTTP handler should be called to service this request. Likewise, if the URI passed to Create() was file://c:\files\myfile.txt, then the file handler FileWebRequest is called to service the request.
The advantage of using the WebRequest and WebResponse classes, rather than casting the returned object to its specific type, such as HttpWebRequest, is that all operations for issuing the request and handling the response are available in these base classes. That is, a Web page can be retrieved solely by creating a WebRequest object and retrieving the WebResponse or by using WebClient, which builds on WebRequest and WebResponse. Notice that for C#, it is necessary to cast the Web object to its specific type to access properties specific to that protocol (for example, the Connection property of the HttpWebRequest class).
As mentioned earlier, the sequence of calls for handling a request- response operation with WebRequest is the same as it is for the HttpWebRequest class. The following code illustrates this point:
C#
// Create the request object
WebRequest request = WebRequest.Create(address);
// Issue the request
WebResponse response = request.GetResponse();
// Read the content
StreamReader reader = new StreamReader(response.GetResponseStream(), Encoding.ASCII);
string content = reader.ReadToEnd();
// Display the content to the console
Console.WriteLine(content);
// Close the response
response.Close()
Visual Basic .NET
' Create the request object
Dim request As WebRequest = WebRequest.Create(address)
' Issue the request
Dim response As WebResponse = request.GetResponse()
' Read the content
Dim reader As New StreamReader(response.GetResponseStream(), Encoding.ASCII)
Dim content As String = reader.ReadToEnd()
' Display the content to the console
Console.WriteLine(content)
' Close the response
response.Close();
Cookies
Cookies are used by Web servers to associate information with a particular user so that context information can be maintained for a given user across multiple Web requests. When a Web request is received by the server, the server might include a cookie (via the Set-Cookie header) for the client to include in each subsequent request to that server. This way, each subsequent request to the server will include the cookie information in the Cookie header.
A cookie itself is simply a collection of name/value strings. Cookies are defined in RFC 2109 and RFC 2965. A single name/value pair is exposed in the .NET Framework through the Cookie class while the CookieCollection class stores multiple Cookie objects as a single store. A Cookie has several properties besides the name and value (expiration date, for example). The following code creates two simple cookies and adds them to a CookieCollection object.
C#
Cookie cookieObj;
CookieCollection cookieJar = new CookieCollection();
cookieObj = new Cookie("user", "Joe");
cookieJar.Add( cookieObj );
cookieObj = new Cookie("item", "ISBN0123456789");
cookieJar.Add( cookieObj )
Visual Basic .NET
Dim cookieObj As Cookie
Dim cookieJar As CookieCollection = New CookieCollection
cookieObj = New Cookie("user", "Joe")
cookieJar.Add(cookieObj)
cookieObj = New Cookie("item", "ISBN0123456789")
cookieJar.Add(cookieObj)
|
The cookies associated with a Web request are exposed through the CookieContainer property of the HttpWebRequest class, which is of the type CookieContainer. Before issuing the request, the CookieContainer can be set by instantiating a new instance of a CookieContainer with either a single Cookie object or a CookieCollection. The cookie values in the container will then be a part of the Web request as the cookie header. Notice that the CookieContainer object is a collection and can be accessed through array index notation. After the client issues a Web request, the server sends a response received by the client as an HttpWebResponse object. The cookies set by the server are available through the Cookies property, which is a CookieContainer object.
Asynchronous HTTP Model
As we’ve seen with most other classes, the Web request-response classes (both the HTTP-specific classes as well as the generic Web classes) offer an asynchronous I/O pattern for retrieving the request stream and the response. The asynchronous Web pattern is useful when issuing multiple Web requests concurrently from a single application as well as for maximizing performance. |
|
The first asynchronous method is BeginGetRequestStream(), which is used for HTTP methods other than GET and HEAD. When this asynchronous request is issued, the callback is invoked and the request stream can be retrieved from the request object to send data associated with the Web request. If BeginGetRequestStream() is invoked on a Web request whose method is GET or HEAD, a ProtocolViolationException exception is thrown. The BeginGetRequestStream() is most commonly used with the POST method, as shown in the following example:
C#
// Common state information
public class HttpState
{
public HttpWebRequest httpRequest;
public HttpWebResponse httpResponse;
}
public class AsyncPost
{
public void DoAsyncPost()
{
try
{
HttpState httpRequestState = new HttpState();
// Setup the request
httpRequestState.httpRequest = (HttpWebRequest)WebRequest.Create("http://www.microsoft.com/");
httpRequestState.httpRequest.Method = "POST";
httpRequestState.httpRequest.ContentType = "application/x-www-form-urlencoded";
// Post the async operation
IAsyncResult ar = httpRequestState.httpRequest.BeginGetRequestStream(
new AsyncCallback(HttpRequestStreamCallback), httpRequestState);
// Process the response stream
httpRequestState.httpResponse = (HttpWebResponse)httpRequestState.httpRequest.GetResponse();
// Retrieve the response stream
StreamReader responseStream = new StreamReader(httpRequestState.httpResponse.GetResponseStream());
// Receive the response
httpRequestState.httpResponse.Close();
}
catch (Exception ex)
{
Console.WriteLine("Exception occurred: {0}", ex.Message);
}
}
private static void HttpRequestStreamCallback(IAsyncResult ar)
{
try
{
// State of request is set to asynchronous.
HttpState httpRequestState = (HttpState)ar.AsyncState;
// End of the Asynchronous request.
Stream streamResponse = httpRequestState.httpRequest.EndGetRequestStream(ar);
byte[ ] postData = new byte[1024];
// Setup data to write on the request stream
streamResponse.Write(postData, 0, postData.Length);
streamResponse.Close();
}
catch (Exception ex)
{
Console.WriteLine(ex.ToString());
}
}
}
Visual Basic .NET
' Common state information
Public Class HttpState
Public httpRequest As HttpWebRequest
Public httpResponse As HttpWebResponse
End Class
Public Class AsyncPost
Public Sub DoAsyncPost()
Try
Dim httpRequestState As HttpState = New HttpState
' Setup the request
httpRequestState.httpRequest = WebRequest.Create("http://www.microsoft.com/")
httpRequestState.httpRequest.Method = "POST"
httpRequestState.httpRequest.ContentType = "application/x-www-form-urlencoded"
' Post the async operation
Dim ar As IAsyncResult = httpRequestState.httpRequest.BeginGetRequestStream(_
AddressOf HttpRequestStreamCallback, httpRequestState )
' Process the response stream
httpRequestState.httpResponse = httpRequestState.httpRequest.GetResponse()
' Retrieve the response stream
Dim responseStream As StreamReader = New StreamReader(httpRequestState.httpResponse.GetResponseStream())
' Receive the response
httpRequestState.httpResponse.Close()
Catch ex As Exception
Console.WriteLine("Exception occurred: {0}", ex.Message)
End Try
End Sub
Private Sub HttpRequestStreamCallback(ByVal ar As IAsyncResult)
Try
' State of request is set to asynchronous.
Dim httpRequestState As HttpState = ar.AsyncState
' End of the Asynchronous request.
Dim streamResponse As Stream = httpRequestState.httpRequest.EndGetRequestStream(ar)
Dim postData(1024) As Byte
' Setup data to write on the request stream
streamResponse.Write(postData, 0, postData.Length)
streamResponse.Close()
Catch ex As Exception
Console.WriteLine(ex.ToString())
End Try
End Sub
End Class
The second asynchronous method is BeginGetResponse(), which invokes the callback once the request has been sent to the server and the response headers received. Within the callback, the application can retrieve the entity by first retrieving the Web response via the GetResponseStream() method followed by retrieving the response stream. The following code shows how to issue an asynchronous BeginGetResponse(), as well as the asynchronous delegate method.
C#
public class AsyncGet
{
public void DoAsyncGet()
{
try
{
HttpState httpStateRequest = new HttpState();
httpStateRequest.httpRequest = (HttpWebRequest)WebRequest.Create("http://www.microsoft.com/");
// Get the response object
IAsyncResult ar = httpStateRequest.httpRequest.BeginGetResponse(new AsyncCallback(HttpResponseCallback), httpStateRequest);
}
catch (WebException wex)
{
Console.WriteLine("Exception occurred on request: {0}", wex.Message);
}
}
private static void HttpResponseCallback(IAsyncResult ar)
{
try
{
HttpState httpRequestState = (HttpState)ar.AsyncState;
// Complete the asynchronous request
httpRequestState.httpResponse = (HttpWebResponse)httpRequestState.httpRequest.EndGetResponse(ar);
// Read the response into a Stream object.
Stream httpResponseStream = httpRequestState.httpResponse.GetResponseStream();
// Post asynchronous Read operations on stream
return;
}
catch (WebException ex)
{
Console.WriteLine("Exception: {0}", ex.Message);
}
}
}
Visual Basic .NET
Public Class AsyncGet
Public Sub DoAsyncGet()
Try
Dim httpStateRequest As HttpState = New HttpState
httpStateRequest.httpRequest = WebRequest.Create("http://www.microsoft.com/")
' Get the response object
Dim ar As IAsyncResult
ar = httpStateRequest.httpRequest.BeginGetResponse(AddressOf HttpResponseCallback, httpStateRequest)
Catch wex As WebException
Console.WriteLine("Exception occurred on request: {0}", wex.Message)
End Try
End Sub
Private Sub HttpResponseCallback(ByVal ar As IAsyncResult)
Try
Dim httpRequestState As HttpState = ar.AsyncState
' Complete the asynchronous request
httpRequestState.httpResponse = httpRequestState.httpRequest.EndGetResponse(ar)
' Read the response into a Stream object.
Dim httpResponseStream As Stream = httpRequestState.httpResponse.GetResponseStream()
' Post asynchronous Read operations on stream
Return
Catch ex As WebException
Console.WriteLine("Exception: {0}", ex.Message)
End Try
End Sub
End Class
It is possible to cancel an asynchronous Web request after it is issued. This can be desirable if the operation is taking too long, such as with a heavily burdened Web server. Cancellation is done by calling the Abort() method on the Web request object. There are a number of ways to structure an application to handle timing out a Web request and aborting the connection. A simple method is to use the thread pool to schedule a timer event using the WaitOrTimerCallback() and specifying the context information for the request along with the timeout length, as the following code illustrates:
C#
public void DoAsyncWithTimeout()
{
HttpState httpRequestState = new HttpState();
// Create the request
httpRequestState.httpRequest = (HttpWebRequest)WebRequest.Create("http://www.microsoft.com/");
// Post the async get response
IAsyncResult ar = httpRequestState.httpRequest.BeginGetResponse(new AsyncCallback( HttpResponseCallback ), httpRequestState);
// Register for a timeout
ThreadPool.RegisterWaitForSingleObject(
ar.AsyncWaitHandle,
new WaitOrTimerCallback( RequestTimeoutCallback ),
httpRequestState,
10 * 1000, // 10 second timeout
true
);
}
static void RequestTimeoutCallback( object state, bool timedout )
{
if ( timedout == true )
{
HttpState requestState = state as HttpState;
if ((requestState != null) && (requestState.httpRequest != null))
{
requestState.httpRequest.Abort();
}
}
}
Visual Basic .NET
Public Sub DoAsyncWithTimeout()
Dim httpRequestState As HttpState = New HttpState
' Create the request
httpRequestState.httpRequest = WebRequest.Create("http://www.microsoft.com/")
' Post the async get response
Dim ar As IAsyncResult = _
httpRequestState.httpRequest.BeginGetResponse( _
AddressOf HttpResponseCallback, _
httpRequestState _
)
' Register for a timeout
ThreadPool.RegisterWaitForSingleObject( _
ar.AsyncWaitHandle, _
AddressOf RequestTimeoutCallback, _
httpRequestState, _
10 * 1000, _
True)
End Sub
Private Sub RequestTimeoutCallback(ByVal state As Object, ByVal timedout As Boolean)
If timedout = True Then
Dim requestState As HttpState = state
If (Not requestState Is Nothing) And (Not requestState.httpRequest Is Nothing) Then
requestState.httpRequest.Abort()
End If
End If
End Sub
In the preceding code sample, if the asynchronous delegate registered with the BeginGetResponse() call does not fire within 10 seconds, the RequestTimeoutCallback() delegate will be invoked. Notice that the state information associated with the asynchronous GET request is also passed to the timeout function. This context block should contain the HttpWebRequest object so that the Abort() method can be called on it. Note that due to the way asynchronous methods use a thread pool, it is not recommended that you mix synchronous and asynchronous method calls. This is because calling from within an asynchronous delegate will block the thread, which leaves fewer threads to service other asynchronous delegates, leading to potential starvation. For example, if your application calls HttpWebResponse.BeginGetResponseStream() to retrieve the stream, it should then use the Stream.BeginRead() method on the stream rather than Stream.Read().