Tuesday, September 20, 2011

Abnormal mdrv termination


The following are pointers on how to go about debugging the “Abnormal mdrv termination error”

1. Hyper threading – Check whether Hyperthreading is enabled on the Operating System. Currently Loadrunner does not support it. You can use the following tool:
 xxxxxxxx

2. The System resources play a very important role during the Load test.
     a. CPU utilization/ Memory utilization – Check the CPU utilization of the Load
         Generator machines. It is recommended that it is always below 85%. Try running
         with fewer vusers on the Load generator machines and monitor the CPU usage
         with increase in load
     b. Temporary directory space – Ensure that the temporary directory (i.e. Environment 
variables -> TEMP and TMP) point to a location that has enough disk space. During a load test Load generators will write into the %Temp% folder and these files do not get deleted at the end of the Load test. These files need to be manually deleted. Ensure that this directory has enough space.

3. Check whether the scripts are running as a thread/process in Controller-> Runtime
    Settings-> Miscellaneous-> Multithreading. Every protocol has a default value for the
    number of threads one mdrv driver can  spawn. This value can be specified in the
    <Loadrunner>\dat\protocols\xxx.lrp.
 For example: For a web script the mdrv can spawn 50 threads.
 For Citrix: 10 threads.
 If this value needs to be increased/decreased the following line should be  
 added/changed for the particular protocol xxx.lrp file:

 [VuGen]
 MaxThreadPerProcess=10 

 Previously it was possible to allocate the threads by adding in the wlrun7.ini

 MaxThreadPerDriver=<number of threads to spawn>      

 But the above mentioned change in the wlrun7.ini is no longer valid. It is OBSOLETE

4. Disable any Anti-virus running on the system. It may interrupt the working of the
    Controller. Check for any Windows Update scheduled during the time of the load test
    and disable it. A good debugging tool would be to check the event viewer to see if any
    scheduled tasks/applications have started up that might have interrupted the Load test.

5. Check if the “Enable IP Spoofer” is checked or “Web page breakdown” is enabled. If   
    there are any communication problems due to IP spoofing it may ultimately result in an
    abnormal termination. For SSL applications we have seen problems when the Web
    Page Breakdown is enabled.

6. Ask for the scripts for investigation. If the script contains any custom C Code that has
    memory allocations there are possibilities of memory leaks. Check to see that all
    memory leaks have been freed. This could result in inefficient use of resources during
    load test

7. If the load test results in a large number of errors/second the output window will be
    unable handle this and result in an error. There are 2 options:
     a.  Go to Controller->Runtime Settings->Log-> Uncheck “Enable logging”
     b.  However the above option will completely block all errors. If you need some of 
          the errors to be logged you can split the vusers between 2 groups
For example if running a 500 vuser load test, create 1 group of 100 and one of 400. In the Controller->Runtime Settings->Log-> check “Enable logging” when “Error occurs” 
For the group with 400 vusers go to the command line and disable logging for this group.
Controller-> Details-> More-> Command Line:  -disable_ message
This will generate errors only for 100 vusers but block any messages for the remaining 400 vusers. This will reduce the rate of errors while still giving you error messages for debugging

8. On running a web services script that has soap_request() callsin Controller, notification  
    messages are generated that keep increasing the size of the output.mdb file. Over a  
    large load test this file can go over 2GB ( which is the limit of MS Access) This will
    cause the wlrun.exe to consume a very high amount of memory and eventually crash
    and even not be able to collate data since the output file is too large. Current
    workarounds are:
    a. In the wlrun7.ini file
        [output]
        ExportMessagesToFile=1 ( Which saves the output in a text file instead of mdb)
    b. Block any output completely by adding to wlrun7.ini
        [output]
        BlockOutputMessages=1
    c. Refer 7.b

9. On running multiple GUI (Winrunner/QTP) scripts using Terminal sessions we get  
     “Abnormal termination..” errors when using the “Terminal Services Manager” feature
     ( LR 7.8FP1 and LR 8.0). On using the LG:0, LG:1 to connect to the sessions
     individually it works fine. While using the terminal services manager feature,
     Loadrunner tries to run the script on the same session (agent process). There is a
     working patch for Winrunner but for QTP the patch is still to be tested

10. Winrunner 8.0/ Loadrunner 8.0 integration still needs the lr_wr_patch. Although the
      patch Readme document specifies only Winrunner 7.5/7.6 we can use the same patch
      for Winrunner 8.0 too.

11. If script runs fine in Vugen but fails for 1 user on localhost in Controller, compare the
      command line arguments of both runs to see the differences. For the VuGen the


      command line arguments can be obtained from the mdrv_cmd.txt in the script folder.
      in controller the command line arguments can be obtained from
      process_mdrv_cmdln.prm  in the brr_xxxx folder in the %temp% directory.

12. Check for  issues with LG-Controller connectivity due to multiple NICs or NIC
      limitation (queue length)

13.  On running a scenario where multiple scripts access a common parameter file i.e. the
       parameter file is stored in a shared location, there could be a sharing violation and
       lead to an abnormal program termination. In order to eliminate this error, have a 
       copy of the parameter file in each of the script folders. This will prevent the
       parameter file being locked for access by other scripts.

No comments:

Post a Comment