Merging multiple EMC NAR files

on

For my EMC SAN I have a script that exports the performance data (NAR files) to a folder every hour. This allows me to easily review the files at a later time and even import them into Excel or other tools.
The problem comes when attempting to merge a large number of files for reviewing a long time period (several weeks or months). The EMC Navisphere UI has a function to merge two NAR files, but with hundreds or thousands of files, this is impossible.
I did some research and found that the Navisphere CLI can combine NAR files from command line. The basic syntax is as follows:
NaviSECCli.exe analyzer -archivemerge -data file1.nar file2.nar -out file3.nar
Using this command, I created a script that used the following logic:

  1. Rename the first file as Temp.Nar
  2. Begin loop though all the files
  3. Merge the current file with Temp.Nar, creating Output.Nar
  4. Rename Output.Nar to Temp.Nar
  5. Continue with loop

The only problem I found with this merge process is that it becomes slower as the files increase in size. So if you continually add to a single file (like I did), it will start out fast and gradually come to a crawl. This is fine for merging a few dozen files, but not the thousands that I was running into.
I reworked my script and came up with the following logic:

  1. Create an empty variable named tempFile
  2. Begin loop though all the files
  3. If tempFile is empty
    1. Assign it the current file
    2. Continue with loop
  4. If tempFile isn’t empty
    1. Merge the current file and tempFile, creating a file with a random name
    2. Continue with loop
  5. Repeat loop until only 1 file remains

While this may seem slower at first glance, its faster because it works with small files more frequently. Only after multiple passes does the merge process begin working with larger files that can take several minutes to complete.

naviPath="C:Program Files (x86)EMCNavisphere CLI"

SET oShell = WScript.CreateObject("Wscript.Shell")
SET objArgs = Wscript.Arguments
folderspec = objArgs(0)

Set fso = CreateObject("Scripting.FileSystemObject")
DO WHILE fso.GetFolder(folderspec).Files.Count >1 
    MergeNAR(folderspec)
Loop


SUB MergeNAR(folderspec)
    dim tempFile
    tempFile=NULL
    Set f = fso.GetFolder(folderspec)
    Set fc = f.Files

    For Each f1 in fc
        IF IsNull(tempFile) THEN
            SET tempFile = f1
        ELSE
            strCmd = "'" & naviPath & "NaviSECCli.exe' analyzer -archivemerge -data '" & folderspec & "" & f1.name & "' '" & folderspec & "" & tempFile.Name & "' -out '" & folderspec & "" & getTimeStamp & ".nar'"
            strCmd =  Replace(strCmd,"'","""")
            wscript.echo strCmd
            oShell.Run strCmd, 1, true
            f1.Delete
            tempFile.Delete
            tempFile = NULL

        END IF
    Next
END SUB


Function getTimeStamp()
        Dim intSeconds, intMilliseconds, strMilliseconds, intDatePart, intTimePart
        
        intSeconds = (Hour(Now) * 3600) + (Minute(Now) * 60) + Second(Now)
        intMilliseconds = Timer() - intSeconds
        intMilliseconds = Fix(intMilliseconds * 100)
        
        intDatePart = (Year(Now) * 10000) + (Month(Now) * 100) + Day(Now)
        intTimePart = (Hour(Now) * 1000000) + (Minute(Now) * 10000) + (Second(Now) * 100) & "." & intMilliseconds
        
        getTimeStamp = intDatePart & intTimePart 
End Function

Comments

Post a Comment