Friday, July 15, 2011

Multi-threaded Parallel Processing

Many times you come across a situation when you have to execute the same logic across different clients or batch. While running them in simplest manner without multi-threading, your records will be sequentially processed means your process will take longer time to finish. 

You can speed up the above process by implementing multi-threaded parallel processing in few lines of code. To keep it simple in this example, you have to make sure that you have distinct batches or set of records which will not interfere or overlap during this processing. Here is how you can implement in quickest possible manner, make sure your machine has enough processing and threading capabilities.

Step 1

Create a property of method so that you can dynamically set the no of threads if needed

Private ReadOnly Property MaxAllowableThreadCount() As Integer
        Get
            'You can define this value in config file 
            Dim maxThreadCount As Integer = 15
            Return maxThreadCount
        End Get
    End Property

Step 2

Following is the example of multi threading or parallel processing code. Assuming each client will have unique set of records.

Private Function ProcessAllClients() As Integer
      Dim clientList As New List(Of String)
      Dim exceptionMessages As New List(Of String)
      Dim totalNoOfRecProcessed As Integer = 0
 
      'Add some Test Clients
      clientList.Add("A1")
      clientList.Add("A2")
      clientList.Add("A3")
 
 
      If clientList IsNot Nothing AndAlso clientList.Count > 0 Then
          'Customize No of Threads you may want to use as per need or you can simply use a value
          Dim noOfThreadCounts As Integer = MaxAllowableThreadCount
          Dim noOfClients As Integer = clientList.Count
          Dim noOfThreadsGroup As Integer = 0
 
          'Adjust No of Loops as per max allowable thread
          If (noOfClients Mod noOfThreadCounts = 0) Then
              noOfThreadsGroup = CInt(Math.Floor(noOfClients / noOfThreadCounts))
          Else
              noOfThreadsGroup = CInt(Math.Floor(noOfClients / noOfThreadCounts)) + 1
          End If
 
          'Set clients in each group/loop 
          For threadGroup As Integer = 0 To noOfThreadsGroup - 1
              Dim processingTaskCounter As Integer = noOfThreadCounts
 
              If threadGroup = (noOfThreadsGroup - 1) AndAlso (noOfClients Mod noOfThreadCounts > 0) Then
                  processingTaskCounter = noOfClients Mod noOfThreadCounts
              End If
 
              Dim allProcessingTasks(processingTaskCounter - 1) As Task(Of Integer)
              Dim clinetsToProcess As New List(Of String)
              Dim taskCounter As Integer = 0
 
              For i = 0 To processingTaskCounter - 1
                  clinetsToProcess.Add(clientList(CInt(threadGroup * noOfThreadCounts) + i))
              Next
 
              For Each clientCode As String In clinetsToProcess
                  Try
                      Dim currentClient As String = clientCode
                      allProcessingTasks(taskCounter) = New Task(Of Integer)(Function()
                                                                                 Dim pHelper As New ProcessHelper
                                                                                 Return pHelper.ClientProcessingLogic(currentClient)
                                                                             End Function)
                      taskCounter = taskCounter + 1
 
                  Catch ex As Exception
                      exceptionMessages.Add("Sorry! Error Occurred." & ex.Message)
                      totalNoOfRecProcessed = 0
                  End Try
              Next
 
              'Now Execute all client's tasks as once but in seperate thread
              Array.ForEach(allProcessingTasks, Sub(tx) tx.Start())
 
              'Wait till all finishes processing
              Task.WaitAll(allProcessingTasks)
 
              For Each tsk In allProcessingTasks
                  Select Case True
                      Case tsk.IsCanceled
                          exceptionMessages.Add("Task ID" & tsk.Id & " was canceled.")
                      Case tsk.IsFaulted
                          exceptionMessages.Add("Task ID" & tsk.Id & " was faulted.")
                      Case tsk.IsCompleted
                          totalNoOfRecProcessed += tsk.Result
                      Case Else
                          exceptionMessages.Add("Task ID" & tsk.Id & " was in unknown status.")
                  End Select
              Next
          Next
      End If
      Return totalNoOfRecProcessed
  End Function

Step 3

Call the Step 2 from wherever you want to call it

Dim allItemsProcessedCount As Integer = ProcessAllClients()

Now you can see the difference in processing time and no of records processed during one call.

Wednesday, June 15, 2011

XDocument Tips

XDocument was introduced with LINQ to manipulate and query XML Document with ease. Here are some useful tips.

1. Create a simple XDocument

Dim myDoc As XDocument = <?xml version="1.0" encoding="utf-8"?><MyDocument></MyDocument>

2. Merge multiple XDcouments in one with variables and formats

Dim xDocFooter As XDocument = Nothing
Dim docName As String = "XDocument"
Dim docDate As Date = Now
 
       xDocHeader = <?xml version="1.0" encoding="utf-8"?>
                    <DocumentHeader>
                        <DocID>1</DocID>
                        <DocName><%= docName %></DocName>
                        <DocDesc>Test Document 1</DocDesc>
                        <DocDate><%= If(docDate > Date.MinValue, Format(docDate, "yyyy-MM-dd"), "1900-01-01") %></DocDate>
                    </DocumentHeader>
 
       xDocFooter = <?xml version="1.0" encoding="utf-8"?>
                    <DocumentFooter>
                        <FooterName>Footer 1</FooterName>
                        <FooterDesc>End Of Document</FooterDesc>
                    </DocumentFooter>
       myDoc.Root.Add(xDocHeader.Elements)
       myDoc.Root.Add(xDocFooter.Elements)

3. Load XML file as XDocument

Dim xdoc As XDocument = Nothing
xdoc = XDocument.Load("C:\TestXML.xml", LoadOptions.PreserveWhitespace)

4. Read XDocument from/to XML String

Dim xmlString As String = "<MyDocument>Test Doc</MyDocument>"
xdoc = XDocument.Parse(xmlString, LoadOptions.PreserveWhitespace) 'Loading from XML String
xmlString = xdoc.ToString() 'To XML String

5. Read XDocument Elements values

If myDoc.Root.Element("DocumentHeader") IsNot Nothing AndAlso myDoc.Root.Element("DocumentHeader").Element("DocName") IsNot Nothing Then
   docName = myDoc.Root.Element("DocumentHeader").Element("DocName").Value
End If
 
If myDoc.Root.Element("DocumentHeader") IsNot Nothing AndAlso myDoc.Root.Element("DocumentHeader").Element("DocDate") IsNot Nothing Then
   docDate = myDoc.Root.Element("DocumentHeader").Element("DocDate").Value
End If

6. Add New Or Modify Existing XDocument Element’s value

myDoc.Root.Element("DocumentHeader").Add(New XElement("PageNo", "1"))
myDoc.Root.Element("DocumentHeader").Element("DocDesc").Value = "New Test Document Description"

Tuesday, November 23, 2010

LINQ Group By Examples

These are 2 samples of  Group By Examples, one is very simple one but not simplest, second one is little bit intermediate. 

First creating a class to use in the given example

   1: Public Class TestRecord
   2:     Property ID As String
   3:     Property Value As String
   4:     Property RecordDate As Date
   5:  
   6:     Sub New()
   7:  
   8:     End Sub
   9:  
  10:     Sub New(ByVal id As String, ByVal value As String, ByVal recDate As Date)
  11:         Me.ID = id
  12:         Me.Value = value
  13:         Me.RecordDate = recDate
  14:     End Sub
  15: End Class

 

Now, let say use a button and Gridview to present the data on a page.

   1: Protected Sub Button4_Click(ByVal sender As Object, ByVal e As EventArgs) Handles Button4.Click
   2:         Dim showSimpleGroup As Boolean = False
   3:  
   4:         Dim lstItems As New List(Of TestRecord)
   5:         lstItems.Add(New TestRecord("1", "100", Now))
   6:         lstItems.Add(New TestRecord("2", "100", Date.MinValue))
   7:         lstItems.Add(New TestRecord("3", "100", #2/2/1980#))
   8:         lstItems.Add(New TestRecord("4", "200", Date.MinValue))
   9:         lstItems.Add(New TestRecord("5", "200", #1/1/2010#))
  10:         lstItems.Add(New TestRecord("6", "300", Now))
  11:  
  12:         If showSimpleGroup Then
  13:             Dim recList = From r In lstItems Group By r.Value Into g = Group Select New With {g, .ID = g.Max(Function(p) p.ID), .Value = g.Max(Function(p) p.Value)}
  14:             Me.gvwTest.DataSource = recList
  15:             Me.gvwTest.DataBind()
  16:         Else
  17:             Dim tempList = From d As TestRecord In lstItems Group By d.Value Into g = Group
  18:                             Select New With {g, .ID = g.Max(Function(d) d.ID), .RecordDate = g.Max(Function(d) IIf(d.RecordDate = Date.MinValue, Now, d.RecordDate))}
  19:  
  20:             If tempList IsNot Nothing AndAlso tempList.Count > 0 Then
  21:                 Dim recList = From r As TestRecord In lstItems Join t In tempList On t.ID Equals r.ID Select r
  22:                 Me.gvwTest.DataSource = recList
  23:                 Me.gvwTest.DataBind()
  24:             End If
  25:         End If
  26:     End Sub

 

The first grouping will display the distinct value excluding Record Date , second one will use the record’s max date to display the record the distinct value.

Friday, November 05, 2010

WCF Service Hosted in Web Farm or Multi-Homed Scenario

If a WCF Service is deployed in a load balanced environment and hosted on multiple Web Servers under IIS, which is accessed on a public domain, WCF WSDL document refers to a the responding machine name which is generally inaccessible on a public domain. So what you should be doing to make it right?

As per Microsoft, this design was initially designed in WCF but later when Microsoft went into Azure or Cloud Computing, this situation became an issue and realized the need to fix this problem. So in order to fix this problem, you need to perform two simple steps

1. Download the following Fix from Microsoft and Install on respective machines

                         http://code.msdn.microsoft.com/KB971842/Release/ProjectReleases.aspx?ReleaseId=3228

2. Add the following sections in behavior section of configuration

   1: <serviceBehaviors>
   2:    <behavior name="<name>">
   3:      <useRequestHeadersForMetadataAddress>
   4:        <defaultPorts>
   5:           <add scheme="http" port="80" />
   6:           <add scheme="https" port="443" />
   7:         </defaultPorts>
   8:       </useRequestHeadersForMetadataAddress>
   9:    </behavior>
  10: </serviceBehaviors>
  11:  
  12:  

For more information, you can checkout the following articles

a. http://support.microsoft.com/kb/980588

b. http://support.microsoft.com/kb/971842

Saturday, October 23, 2010

Using IFRAME in ASP.NET

If you are using an IFRAME in ASP.NET and setting the height and width and not working as per your need. Spent couple of hours trying all possible options and found a very easy solution

1. Remove following line from Page

<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">
2. How to Set values Dynamically

<iframe id="Testframe" runat="Server" />

<script language="vb" runat="server">

 
Protected WithEvents frame1 As System.Web.UI.HtmlControls.HtmlGenericControl
  
  Sub Page_Load()
      Dim frame1 As HtmlControl = CType(Me.FindControl("TestFrame"), HtmlControl)
      With frame1
          .Attributes("src") = "TestPage.aspx"
          .Attributes("width") = "100%"
          .Attributes("height") = "100%"
      End With
      
  End Sub
 
</script>

Wednesday, October 20, 2010

Reading/Uploading Image data using base64 string

How will you upload an image from your local drive without using File Upload control or you do not want to click browse button in order to select and upload the image file. Here is an alternate solution. I came across a situation in which I had to upload an Image file from local drive without using server side File Upload control already available in .NET.

 Client Side Code

The following example illustrates a VBScript Client side function using ADODB Stream and MSXML2 DOMDocument Object.

   1: <script language="vbscript" type="text/vbscript">
   2:  
   3:    'fileName can be any file including full path on your local drive
   4:  
   5:     Function PostImageData(ByVal fileName)
   6:        Const fsDoOverwrite     = true  ' Overwrite file with base64 code
   7:        Const fsAsASCII         = false ' Create base64 code file as ASCII file
   8:        Const adTypeBinary      = 1     ' Binary file is encoded
   9:        Dim objXML,objDocElem
  10:        Dim objStream, objFSO
  11:                
  12:        Set objXML = CreateObject("MSXml2.DOMDocument")
  13:        Set objDocElem = objXML.createElement("Base64Data")
  14:  
  15:        On Error Resume Next
  16:        Set objFSO = CreateObject("Scripting.FileSystemObject")
  17:        Set objStream = CreateObject("ADODB.Stream")
  18:        
  19:        If Err.Number = 0 Then
  20:            If objFSO.FileExists(fileName)  Then
  21:                objStream.Type = adTypeBinary
  22:                objStream.Open()
  23:                objStream.LoadFromFile(fileName)
  24:                objDocElem.dataType = "bin.base64"
  25:                objDocElem.nodeTypedValue = objStream.Read()
  26:                frmPageName.txtBase64Data.value = objDocElem.text                                   
  27:                objStream.Close()
  28:                'objFSO.DeleteFile fileName,true 'Delete the Image file if needed
  29:            Else
  30:                MsgBox("Could not find Image file " & fileName)
  31:            End If
  32:          Else
  33:              MsgBox("Your Browser Secuirty Settings currently do not allow image file reading.")
  34:          End If
  35:  
  36:        Set objXML = Nothing
  37:        Set objDocElem = Nothing
  38:        Set objStream = Nothing
  39:        Set objFSO = Nothing
  40:    End function
  41:  
  42:   </script>

 

Note: In order to make this example work you will need to make sure that following options are enabled in IE

  1. Open IE and Go to Tools –> Internet Options –>Security Tab
  2. Select Trusted Sites –> Sites
  3. Type the website you are planning to run this page and click add and then Close.
  4. Now Click on Custom Level button and make the following options are enabled in ActiveX controls and plug-ins

         a. Allow Previously unused ActiveX Controls to run …

         b. Automatic Prompting for ActiveX controls

         c. Binary and script behaviors

         d. Display video and  animation on a webpage that …

         e. Download signed ActiveX controls

         f. Download unsigned ActiveX controls

         g. Initialize and script ActiveX controls not marked as safe …

         h. Only allow approved domains to use ActiveX without …

          i. Run ActiveX controls and plug-ins

          j. Script ActiveX controls marked safe for scripting

 

5. Make sure following options are set to enabled in Miscellaneous section

         a. Access data sources across domains

         b. Allow META REFRESH

         c. Allow scripting of Microsoft web browser control

         d. Allow script-initiated windows without size or position …

6.  After making all the above changes, click on OK then Apply and then OK. Now Close the browser

 

Server Side Code

   1: Private Function ConvertImageFromBase64String() As Image
   2:         Dim NewImageFile As Image = Nothing
   3:         Try
   4:             Dim bytesData As Byte() = System.Convert.FromBase64String(txtBase64Data.Value)
   5:  
   6:             If bytesData.GetUpperBound(0) > 0 Then
   7:                 Using ImageStream As MemoryStream = New MemoryStream(bytesData)
   8:                     NewImageFile = Image.FromStream(ImageStream, True)
   9:                 End Using
  10:                 NewImageFile.Save(SERVER_PATH, System.Drawing.Imaging.ImageFormat.Bmp)
  11:             Else
  12:                 NewImageFile = Nothing
  13:             End If
  14:         Catch ex As Exception
  15:             NewImageFile = Nothing
  16:         End Try
  17:  
  18:         Return NewImageFile
  19:     End Function

 

Now Save or Present the image wherever you want.