< VB .NET Asynchronous Http Get Request Program Example | Main | C# Service Point Program Example >


 

Chapter 10 Part 12:

HTTP with .NET

 

 

What do we have in this chapter 10 Part 12?

  1. Connection Management

  2. C++ Service Point Program Example

 

Connection Management

 

When retrieving multiple URIs, it is often the case that many of the resources reside on the same host. If an application needs to retrieve hundreds or thousands of URIs, the application should limit the number of concurrent HTTP requests. This allows efficient use of network bandwidth and lessens the load on the Web server. To accomplish this, the .NET Framework offers the ServicePointManager and ServicePoint classes for managing HTTP requests.

The ServicePointManager class defines limits for a destination, such as the maximum number of concurrent connections allowed at any given time. The first step for managing Web connections is to establish the limits on the ServicePointManager that will be applied to subsequent connection requests. Table 10-5 lists the important properties of the ServicePointManager class.

 

Table 10-5: ServicePointManager Properties

 

Property

Description

DefaultConnectionLimit

Maximum number of concurrent connections to a single ServicePoint

MaxServicePointIdleTime

Maximum time in milliseconds a ServicePoint can be idle before being garbage collected

MaxServicePoints

Maximum number of concurrent ServicePoint objects

 

Once the restrictions are set on the ServicePointManager, a ServicePoint object is retrieved for a given server before any Web requests are issued to that server. A ServicePoint is retrieved by calling FindServicePoint(). If a ServicePoint already exists from a previous request, the same object is returned; otherwise, a new ServicePoint is created so long as the MaxServicePoints limit is not exceeded. The following code illustrates this:

 

C#

 

ServicePoint    sp;

 

GlobalProxySelection.Select = new WebProxy("http://myproxy:80/");

 

ServicePointManager.MaxServicePoints        = 2;

ServicePointManager.DefaultConnectionLimit  = 2;

ServicePointManager.MaxServicePointIdleTime = 20000; // 20 seconds

 

sp = ServicePointManager.FindServicePoint(new Uri("http://www.one.com"));

// GetUriResource retrieves a URI

GetUriResource("http://www.one.com/one.html");

 

sp = ServicePointManager.FindServicePoint(new Uri("http://www.two.com"));

GetUriResource("http://www.two.com/two.html")

 

Visual Basic .NET

 

Dim sp As ServicePoint

 

GlobalProxySelection.Select = New WebProxy("http://myproxy:80/")

 

ServicePointManager.MaxServicePoints = 2

ServicePointManager.DefaultConnectionLimit = 2

ServicePointManager.MaxServicePointIdleTime = 20000 ’ 20 seconds

 

sp = ServicePointManager.FindServicePoint(New Uri("http://www.one.com"))

' GetUriResource retrieves a URI

GetUriResource("http://www.one.com/one.html")

 

sp = ServicePointManager.FindServicePoint(New Uri("http://www.two.com"))

GetUriResource("http://www.two.com/two.html");

 

In this example, the application establishes a limit of two concurrent servers (MaxServicePoints) with two concurrent requests to each sever (DefaultConnectionLimit). It then sets www.one.com as the first ServicePoint to be managed and www.two.com as the second managed site. The GetUriResource() function issues an asynchronous HttpWebRequest for the resource.

The service point limit in the example is two, and only two concurrent ServicePoint objects are requested; therefore, the subsequent Web requests succeed. However, if a third call to FindServicePoint was requested to a server (for example, www.three.com) that had not been requested previously, an InvalidOperationException would be thrown because this third ServicePoint would exceed the established limit.

Only a single Web request is issued for each ServicePoint. If three concurrent instances of HttpGetRequest are issued for URIs residing on a single ServicePoint host (such as www.one.com), the first two are allowed while the third is blocked until one of the initial requests completes.

 

C++ Service Point Program Example

 

Create a new CLR console application project and you might want to use ServicePointCP as the project and the solution names.

 

C++ Service Point Program Example - creating a new CLR console application project in Visual Studio 2008 IDE

 

 

 

 

Add the following code that includes two classes and main().

 

using namespace System;

using namespace System::Net;

using namespace System::Collections;

using namespace System::Threading;

using namespace System::IO;

 

/// <summary>

/// Maintains context information for each HTTP request

/// </summary>

public ref class HttpState

{

public:

    HttpWebRequest^ httpRequest;

    HttpWebResponse^ httpResponse;

    Stream^ httpResponseStream;

    Uri^ uriResource;

    array<Byte>^ readBuffer;

    ManualResetEvent^ doneEvent;

    int bytesReceived;

 

public:

    static const int DefaultReadBufferLength = 4096;

 

    /// <summary>

    /// Constructor for initializing HttpState object

    /// </summary>

    /// <param name="resource">URI to retrieve</param>

    /// <param name="done">Event to signal when done</param>

public:

    HttpState(Uri^ resource, ManualResetEvent^ done)

    {

        httpRequest = nullptr;

        httpResponse = nullptr;

        uriResource = resource;

        readBuffer = gcnew array<Byte>(DefaultReadBufferLength);

        doneEvent = done;

        bytesReceived = 0;

    }

};

 

/// <summary>

/// Contains the ServicePoint sample

/// </summary>

public ref class ServicePointClass

{

    /// <summary>

    /// Displays usage information

    /// </summary>

public:

    static void usage()

    {

        Console::WriteLine("Executable_file_name [-u URI] [-p proxy] [-l count] [-s count]");

        Console::WriteLine("Available options:");

        Console::WriteLine("     -u URI      URI resource to retrieve");

        Console::WriteLine("     -p proxy    Proxy server to use");

        Console::WriteLine("     -l count    Maximum number of concurrent connections");

        Console::WriteLine("     -s count    Maximum number of ServicePoint instances");

        Console::WriteLine();

    }

 

    /// <summary>

    /// Asynchronous delegate for read operation on the HTTP stream.

    /// </summary>

    /// <param name="ar">Asynchronous context information for operation</param>

public:

    static void HttpStreamReadCallback(IAsyncResult^ ar)

    {

        HttpState^ httpInfo = (HttpState^)ar->AsyncState;

 

        try

        {

            long percent = 0;

            int count = httpInfo->httpResponseStream->EndRead(ar);

 

            httpInfo->bytesReceived += count;

            percent = (long)(httpInfo->bytesReceived * 100) / (long)httpInfo->httpResponse->ContentLength;

            Console::WriteLine("{0}: read {1} bytes: {2}%", httpInfo->uriResource->AbsoluteUri->ToString(), count, percent.ToString()->PadLeft(3));

 

            if (count > 0)

            {

                PostStreamRead(httpInfo);

            }

            else

            {

                httpInfo->httpResponseStream->Close();

                httpInfo->httpResponseStream = nullptr;

                httpInfo->httpResponse->Close();

                httpInfo->httpResponse = nullptr;

                httpInfo->doneEvent->Set();

            }

        }

        catch (Exception^ ex)

        {

            Console::WriteLine("Exception occurred: {0}", ex->Message);

            if (httpInfo->httpResponseStream != nullptr)

                httpInfo->httpResponseStream->Close();

            if (httpInfo->httpResponse != nullptr)

                httpInfo->httpResponse->Close();

            httpInfo->doneEvent->Set();

        }

    }

 

    /// <summary>

    /// Post as an asynchronous receive operation on the HTTP stream.

    /// </summary>

    /// <param name="httpInfo">Context information for HTTP request</param>

public:

    static void PostStreamRead(HttpState^ httpInfo)

    {

        IAsyncResult^ asyncStreamRead = httpInfo->httpResponseStream->BeginRead(

            httpInfo->readBuffer,

            0,

            httpInfo->readBuffer->Length,

            gcnew AsyncCallback(HttpStreamReadCallback),

            httpInfo

            );

    }

 

 

    /// <summary>

    /// Asynchronous delegate for the HTTP response.

    /// </summary>

    /// <param name="ar">Asynchronous context information</param>

public:

    static void HttpResponseCallback(IAsyncResult^ ar)

    {

        HttpState^ httpInfo = (HttpState^)ar->AsyncState;

 

        try

        {

            httpInfo->httpResponse = (HttpWebResponse^)httpInfo->httpRequest->EndGetResponse(ar);

            httpInfo->httpResponseStream = httpInfo->httpResponse->GetResponseStream();

            PostStreamRead(httpInfo);

        }

        catch (WebException^ wex)

        {

            Console::WriteLine("Exception occurred: {0}", wex->Message);

            Console::WriteLine(wex->StackTrace);

            httpInfo->doneEvent->Set();

        }

    }

 

    /// <summary>

    /// Retrieves the given resource

    /// </summary>   

    /// <param name="uriResource">URI to retrieve</param>

    /// <param name="doneEvent">Event to signal when done</param>

    public:

    static void GetUriResource(Uri^ uriResource, ManualResetEvent^ doneEvent)

    {

        HttpState^ httpInfo = gcnew HttpState(uriResource, doneEvent);

 

        try

        {

            Console::WriteLine("Issuing an async GET request for: {0}", uriResource->ToString());

            httpInfo->httpRequest = (HttpWebRequest^)WebRequest::Create(uriResource->AbsoluteUri);

            httpInfo->httpRequest->BeginGetResponse(gcnew AsyncCallback(HttpResponseCallback), httpInfo);

            Console::WriteLine("POSTED lor!");

        }

        catch (WebException^ wex)

        {

            Console::WriteLine("Exception occurred: {0}", wex->Message);

            doneEvent->Set();

        }

    }

};

 

/// <summary>

/// Main application which parses the command line and issues the HTTP request.

/// </summary>

/// <param name="args">Command line arguments</param>

int main(array<System::String ^> ^args)

{

    ArrayList^ uriList = gcnew ArrayList();

    IWebProxy^  httpProxy = WebRequest::DefaultWebProxy;

    int connectionLimit = 2, servicePointLimit = 1;

    int i;

 

    // Parse the command line

    if(args->Length != 0)

    {

    for(i=0; i < args->Length ;i++)

    {

        try

        {

            if ( ( args[i][0] == '-' ) || ( args[i][0] == '/' ) )

            {

                switch ( Char::ToLower( args[i][1] ) )

                {

                    case 'u':

                        // URI to download (get)

                        uriList->Add( gcnew Uri( args[ ++i ] ) );

                        break;

                    case 'p':

                        // Name of proxy server to use

                        httpProxy = gcnew WebProxy( args[ ++i ] );

                        break;

                    case 'l':

                        // Number of connections limited to service point manager

                        connectionLimit = Convert::ToInt32( args[ ++i ] );

                        break;

                    case 's':

                        // Service point limit

                        servicePointLimit = Convert::ToInt32( args[ ++i ] );

                        break;

                    default:

                        ServicePointClass::usage();

                        return 0;

                }

            }

        }

        catch(Exception^ err)

        {

            Console::WriteLine("Error: " + err->Message);

            ServicePointClass::usage();

            return 0;

        }

    }

}

else

{

    ServicePointClass::usage();

    return 0;

}

 

    try

    {

       array<ManualResetEvent^>^ requestEvents = gcnew array<ManualResetEvent^>(uriList->Count);

       // Setup the ServicePointManager first

       WebRequest::DefaultWebProxy = httpProxy;

       ServicePointManager::MaxServicePoints = servicePointLimit;

       ServicePointManager::DefaultConnectionLimit = connectionLimit;

       ServicePointManager::MaxServicePointIdleTime = 20000; // 20 seconds

       Console::WriteLine("Uri count: {0}", uriList->Count);

       for(i=0; i < uriList->Count ;i++)

       {

            Console::WriteLine("Uri: {0}", ((Uri^)uriList[i])->ToString() );

            String^ baseUri = "http://" + ((Uri^)uriList[i])->Host + "/";

            Console::WriteLine("Base URI: {0}", baseUri );

            ServicePoint^    sp = ServicePointManager::FindServicePoint((Uri^)uriList[i], httpProxy );

            requestEvents[i] = gcnew ManualResetEvent( false );

            ServicePointClass::GetUriResource( (Uri^)uriList[i], requestEvents[i] );

        }

            Console::WriteLine("Waiting...");

            ManualResetEvent::WaitAll( requestEvents );

            Console::WriteLine("Considered done...");

    }

    catch ( Exception^ ex )

    {

            Console::WriteLine("Exception occurred: {0}", ex->Message);

    }

    return 0;

}

 

 

 

 

Build and run the project. The following is the sample output.

 

C++ Service Point Program Example - a sample output

 

The following is the sample output when run from the command prompt with two URIs. There is an error on the percentage of the progress for one of the URI. This is left for your exercise to rectify the logical error!

 

C++ Service Point Program Example - running the project from command prompt with arguments

 


< VB .NET Asynchronous Http Get Request Program Example | Main | C# Service Point Program Example >