«

»

Jun 27 2014

HP HPSA driver 5.x.0.58-1 can cause ESXi 5.x PSOD

A few days ago i received a call from one of my old customers who asked for advice regarding a VMware ESXi 5.5 Purple Screen Of Death (PSOD) situation. He had already created a VMware service request but wanted a chat while waiting for a reply.

I quickly searched the HP support center web site and found out a HPSA driver update is available for their HP Proliant DL380 G7 servers.  The information and release notes, found here (same link can be used for downloading the VIB),  states the following fixed issues for the new HPSA driver:

  • Fixed a memory leak associated with device rescans resulting in out of memory conditions and a potential  PSOD.
  • Fixed a null pointer dereference in error handling code that can cause a PSOD in rare cases when device inquiries fail.
  • Restore LUN numbering policy to start with 1 instead of 0, avoiding potential issues with Raw Device Maps.
  • Enable 64bit DMA mapping instead of default 32bit mapping.
  • Improve null pointer checks in device rescanning code, avoiding a potential PSOD.
  • Restore maximum outstanding command count, removing artificial limitation that could impact performance.
  • Restore support for legacy HP Smart Array P700m controller.

So this is what i advised to upgrade the ESXi hosts HPSA driver to the latest version:

  • Identify the Storage controller you are using and that can easily be done via the vSphere Web Client by going to an ESXi server -> Monitor -> Hardware Status -> Expand the Storage Sensor.Screen Shot 2014-06-27 at 08.04.41
  • Identify existing HPSA driver version by running the following command
    • cat /proc/driver/hpsa/hpsa0 | grep Driver
      Change hpsa number to match your system.
    • The output returned “Driver Version: HP HPSA Driver (v 5.5.0.58-1OEM)” meant the affected software driver was used.
  • Put ESXi host in maintenance mode
  • Copy the VIB to the ESXi host by using the following command
    • scp scsi-hpsa-5.5.0.60-1OEM.550.0.0.1331820.x86_64.vib root@ESXi-FQDN:/tmp
  • Install the VIB by running the following command:
    • esxcli software vib install -v /tmp/scsi-hpsa-5.5.0.60-1OEM.550.0.0.1331820.x86_64.vib –no-sig-check
      The expected output should be:
      Screen Shot 2014-06-27 at 08.28.10
  • Reboot the ESXi hos
  • Verify the latest driver version by running the following command
    • cat /proc/driver/hpsa/hpsa0 | grep Drive
      Change hpsa number to match your system.
  • Exit ESXi host Maintenance mode.

No more PSOD after the HPSA driver upgrade.

18 comments

5 pings

Skip to comment form

  1. Pawel

    You can also try to upgrade P410 firmware to 6.40. I have it on my vsphere 5.5u1 hosts and works without any problems.

  2. magander3

    Thanks for sharing. Still think you should update the driver as well.

    //Magnus

  3. Pawel

    Ofcourse I agree. I do not want to risk PSOD on production.

  4. magander3

    Totally agree.

  5. servers ibm

    It’s awesome designed for me to have a site, which is good for my experience.
    thanks admin

  6. Pawel

    First PSOD on test cluster after me. Driver was updated 5 minutes after PSOD 😉

  7. hp

    Usually I don’t read post on blogs, however I wish to say that this write-up very forced me to take a look at and do it!
    Your writing taste has been surprised me. Thank you, very nice article.

  8. Kristopher

    Hmm is anyone else having problems with the pictures on this blog loading?
    I’m trying to find out if its a problem on my end or if it’s the blog.
    Any feed-back would be greatly appreciated.

  9. magander3

    Hi Kristopher,
    i have tried from 2 computers with 4 different browsers and the pictures loads just fine. Don’t know what causes your problem.

    //Magnus

  10. Kamal

    Thanks for your help this was amazing still helping people today Cudos

  11. magander3

    Great, glad you found it useful

    //Magnus

  12. Kamal

    I have just tried this today to get PSOD to disappear, call me a noob but i am unable to get the files copied from my local machine to the ESXi Host.

  13. Kamal

    Because when i try to install it says it cannot find the file.

    🙁

  14. magander3

    you can browse datastore via vSphere Web Client or vSphere client and upload the file. Another option is to enable ssh and use pscp or scp for uploading the file.
    What command are you using when trying to perform the installation ?

    //Magnus

  15. Kamal

    Hi Magnus,

    First off thanks for the great response times love it!

    I am using the command that you used in your post scp scsi-hpsa-5.5.0.60-1OEM.550.0.0.1331820.x86_64.vib root@ESXi-FQDN:/tmp

    I have uploaded the file to the datastore from the vsphere client , in the root directory however this did not work, do i need to to edit the command to look somthing like this : scp scsi-hpsa-5.5.0.60-1OEM.550.0.0.1331820.x86_64.vib root@ESXi-FQDN:/datastore1 ??

    When connected via SSH, I suppose i need to first copy the VIb file to the tmp folder ? if so what command is needed to complete this ?

    Cheers, greatly appreciated.

    Kamal

  16. magander3

    if you uploaded to /tmp folder the provided command should work “esxcli software vib install -v /tmp/scsi-hpsa-5.5.0.60-1OEM.550.0.0.1331820.x86_64.vib –no-sig-check”
    What error do you get?

    If you want to upload to a datastore you should use root@esxi-host:/vmfs/volumes/datastore-name

    thanks

  17. BoMart

    What is wrong?

    [root@vmw1:/tmp] esxcli software vib install -v
    file:/tmp/scsi-hpsa_6.0.0.120-1OEM.600.0.0.2494585.vib –force
    –no-sig-check –maintenance-mode
    [InstallationError]
    The transaction is not supported: VIB
    HPE_bootbank_scsi-hpsa_6.0.0.120-1OEM.600.0.0.2494585 cannot be live
    installed. VIB
    Hewlett-Packard_bootbank_scsi-hpsa_5.5.0.84-1OEM.550.0.0.1331820 cannot
    be removed live.
    Please refer to the log file for more details.

  18. magander3

    Hi,
    have seen this a few times before and if i remember correctly it had to do with to little available disk space on the ESXi boot drive. Can you make sure you have enough available space on the boot device?

    //Magnus

  1. Newsletter: June 29, 2014 | Notes from MWhite

    […] HP HPSA drive 5.x.0.58-1 can cause ESXi 5.x PSOD This issue, and solution makes a good story.  A reminder that proactive maintenance that includes patching is a good idea.  I like how it […]

  2. Good nuggets from the Veeam community forum digest | MSMVPs.com Latest Blog Posts

    […] you are using HP Proliant DL380 G7′s for your vSphere 5.5 hosts, be aware of this potential PSOD issue and patch your hosts soon with this fix from […]

  3. Tweet: Here is the associated background to the #HP PSOD… | Talking IT

    […] is the associated background to the #HP PSOD issue vcdx56.com/2014/06/27/hp-… This info was published by @gostev in the @veeam Digest 00 […]

  4. VMware ESXi 5.5 u1 PSOD on HP ProLiant G7 PSOD | Educational Materials for Technicians, Created by Technicians

    […] must also be noted that Nick Furnell posted a tweet to this blog post which details the same fix, at around the same time HP gave me the wrong driver to fix the issue, […]

  5. Update Manager Disabled in vSphere after 5.5 Update | Talking IT

    […] fix for this issue is clearly explained in this URL BUT although the notes are very clear, it does rely on you using the command line and downloading […]

Leave a Reply

Your email address will not be published. Required fields are marked *

You may use these HTML tags and attributes: <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code class="" title="" data-url=""> <del datetime=""> <em> <i> <q cite=""> <s> <strike> <strong> <pre class="" title="" data-url=""> <span class="" title="" data-url="">