Showing posts with label PowerShell. Show all posts
Showing posts with label PowerShell. Show all posts

Friday, 2 July 2021

Convert Specific Table of Excel Sheet to JSON



There is an excellent script available on GitHub which helps in converting an excel sheet to JSON. The table which it converts to JSON is from the start of the page i.e. from A1 cell (as shown in image below).

I had a little different requirement. I had to convert a specific table among various tables available within a sheet in an excel file as shown in image below.

In example given above, there are multiple sheets within an excel file. The sheet that I am interested in is “Science”. Inside Science sheet there are multiple tables like Class1, Class2 …. Class 10 and so on. And I am interested in finding out Class 6 students data which is at row 44. Header is at row 45 and data starts from row 46. The tables can be at any location (any column and any row) within the sheet. The only identifier I am using is TableName which is “Class 6” in this example.

And this is how I have achieved it -

Input Parameters

The script accepts 3 parameters

$InputFileFullPath: - This is path of input excel file. Example - "C:\Data\ABCDSchool.xlsx"

$SubjectName: - This is name of the sheet inside excel file. Example - "Science"

$ClassName: -This is name of the table within excel sheet. Example - "Class 6"

#Inputs
$InputFileFullPath = "C:\Data\ABCDSchool.xlsx"
$SubjectName = "Science"
$ClassName = "Class 6"

Open excel file and read the “Science” sheet

$excelApplication = New-Object -ComObject Excel.Application
$excelApplication.DisplayAlerts = $false
$Workbook = $excelApplication.Workbooks.Open($InputFileFullPath)

$sheet = $Workbook.Sheets | Where-Object {$_.Name -eq $SubjectName}
if (-not $sheet) {
    throw "Could not find subject '$SubjectName' in the workbook"
}

Grab “Class 6” table within “Science” sheet to work with

$found = $sheet.Cells.Find($ClassName) #find the cell where Class name is mentioned
$beginAddress = $Found.Address(0,0,1,1).Split("!")[1]
$beginRowAddress = $beginAddress.Substring(1,2)
$startHeaderRowNumber = [int]$beginRowAddress + 1 #header row starts 1 row after the class name 
$startDataRowNumber = $startHeaderRowNumber + 1 #student data row starts 1 rows after header row
$beginColumnAddress = $beginAddress.Substring(0,1)
$startColumnHeaderNumber = [BYTE][CHAR]$beginColumnAddress - 65 + 1 #ASCII number of column

Extract Header Columns Name (Logical Seat Location, Actual Seat Location, LAN Port #, Monitor Cable Port, Student Name, Student#, Room Type)

$Headers = @{}
$numberOfColumns = 0
$foundHeaderValue = $true
while ($foundHeaderValue -eq $true) {
    $headerCellValue = $sheet.Cells.Item($startHeaderRowNumber, $numberOfColumns+$startColumnHeaderNumber).Text 
    if ($headerCellValue.Trim().Length -eq 0) {
        $foundHeaderValue = $false
    } else {
        $numberOfColumns++
        if($Headers.ContainsValue($headerCellValue))
        {
            #do not add any duplicate column again.
        }
        else
        {            
            $Headers.$numberOfColumns = $headerCellValue
        }
    }
}

Extract Data Rows (Class 6 Student Information Rows)

$results = @()
$rowNumber = $startDataRowNumber
$finish = $false
while($finish -eq $false)
{
    if ($rowNumber -gt 1) {
        $result = @{}        
        foreach ($columnNumber in $Headers.GetEnumerator()) {
            $columnName = $columnNumber.Value
            $cellValue = $sheet.Cells.Item($rowNumber, $columnNumber.Name+($startColumnHeaderNumber-1)).Value2 # student data row, student data column number
            if($cellValue -eq $null)
            {
                $finish = $true
                break;
            }
            $result.Add($columnName.Trim(),$cellValue.Trim())
        }
        if($finish -eq $false)
        {
            $result.Add("RowNumber",$rowNumber) #adding excel sheet row number for validation        
            $results += $result
            $rowNumber++
        }
    }
}

Create JSON file and close excel file

$inputFileName = Split-Path $InputFileFullPath -leaf
$jsonOutputFileName = "$($inputFileName.Split(".")[0])-$SubjectName-$ClassName.json"
$jsonOutputFileFullPath = [System.IO.Path]::GetFullPath($jsonOutputFileName) #Output file name will be "ABCDSchool-Science-Class 6.json" 


Write-Host "Converting sheet '$SubjectName' to '$jsonOutputFileFullPath'"
$ignoreOutput = $results | ConvertTo-Json | Out-File -Encoding ASCII -FilePath $jsonOutputFileFullPath
$ignoreOutput = $excelApplication.Workbooks.Close()
$ignoreOutput = [void][System.Runtime.InteropServices.Marshal]::ReleaseComObject($excelApplication) 
Full code goes like this
param 
(
    [Parameter(Mandatory=$true)]
    [string]$InputFileFullPath, #excel name
    [Parameter(Mandatory=$true)]
    [string]$SubjectName, #sheet name
    [Parameter(Mandatory=$true)]
    [string]$ClassName #identifier for the table
)

#open excel file
$excelApplication = New-Object -ComObject Excel.Application
$excelApplication.DisplayAlerts = $false
$Workbook = $excelApplication.Workbooks.Open($InputFileFullPath)

#find sheet
$sheet = $Workbook.Sheets | Where-Object {$_.Name -eq $SubjectName}
if (-not $sheet) {
    throw "Could not find subject '$SubjectName' in the workbook"
}

#grab the table within sheet to work with
$found = $sheet.Cells.Find($ClassName) #find the cell where Class name is mentioned
$beginAddress = $Found.Address(0,0,1,1).Split("!")[1]
$beginRowAddress = $beginAddress.Substring(1,2)
$startHeaderRowNumber = [int]$beginRowAddress + 2 #header row starts 1 row after the class name 
$startDataRowNumber = $startHeaderRowNumber + 1 #student data row starts 1 rows after header row
$beginColumnAddress = $beginAddress.Substring(0,1)
$startColumnHeaderNumber = [BYTE][CHAR]$beginColumnAddress - 65 + 1 #ASCII number of column

#Extract Header Columns Name
$Headers = @{}
$numberOfColumns = 0
$foundHeaderValue = $true
while ($foundHeaderValue -eq $true) {
    $headerCellValue = $sheet.Cells.Item($startHeaderRowNumber, $numberOfColumns+$startColumnHeaderNumber).Text 
    if ($headerCellValue.Trim().Length -eq 0) {
        $foundHeaderValue = $false
    } else {
        $numberOfColumns++
        if($Headers.ContainsValue($headerCellValue))
        {
            #do not add any duplicate column again.
        }
        else
        {            
            $Headers.$numberOfColumns = $headerCellValue
        }
    }
}

#Extract Student Information Rows
$results = @()
$rowNumber = $startDataRowNumber
$finish = $false
while($finish -eq $false)
{
    if ($rowNumber -gt 1) {
        $result = @{}        
        foreach ($columnNumber in $Headers.GetEnumerator()) {
            $columnName = $columnNumber.Value
            $cellValue = $sheet.Cells.Item($rowNumber, $columnNumber.Name+($startColumnHeaderNumber-1)).Value2 # student data row, student data column number
            if($cellValue -eq $null)
            {
                $finish = $true
                break;
            }
            $result.Add($columnName.Trim(),$cellValue.Trim())
        }
        if($finish -eq $false)
        {
            $result.Add("RowNumber",$rowNumber) #adding excel sheet row number for validation        
            $results += $result
            $rowNumber++
        }
    }
}

#input excel and output json file name
$inputFileName = Split-Path $InputFileFullPath -leaf
$jsonOutputFileName = "$($inputFileName.Split(".")[0])-$SubjectName-$ClassName.json"
$jsonOutputFileFullPath = [System.IO.Path]::GetFullPath($jsonOutputFileName)

#create json file and close excel file
Write-Host "Converting sheet '$SubjectName' to '$jsonOutputFileFullPath'"
$ignoreOutput = $results | ConvertTo-Json | Out-File -Encoding ASCII -FilePath $jsonOutputFileFullPath
$ignoreOutput = $excelApplication.Workbooks.Close()
$ignoreOutput = [void][System.Runtime.InteropServices.Marshal]::ReleaseComObject($excelApplication) 
The output JSON file will look like below
[
    {
        "Room Type":  "Standard",
        "RowNumber":  46,
        "Student Name":  "Alex",
        "Student#":  "RL45",
        "LAN Port #":  "LAN Port 7-8",
        "Logical Seat Location":  "SL 11",
        "Actual Seat Location":  "Seat43",
        "Monitor Cable Port":  "C-D"
    },
    {
        "Room Type":  "Standard",
        "RowNumber":  47,
        "Student Name":  "Alex",
        "Student#":  "RL45",
        "LAN Port #":  "LAN Port 5-6",
        "Logical Seat Location":  "SL 11",
        "Actual Seat Location":  "Seat43",
        "Monitor Cable Port":  "A-B"
    },
    {
        "Room Type":  "Standard",
        "RowNumber":  48,
        "Student Name":  "John",
        "Student#":  "RL47",
        "LAN Port #":  "LAN Port 3-4",
        "Logical Seat Location":  "SL 11",
        "Actual Seat Location":  "Seat43",
        "Monitor Cable Port":  "C-D"
    },
    {
        "Room Type":  "Standard",
        "RowNumber":  49,
        "Student Name":  "John",
        "Student#":  "RL47",
        "LAN Port #":  "LAN Port 1-2",
        "Logical Seat Location":  "SL 11",
        "Actual Seat Location":  "Seat43",
        "Monitor Cable Port":  "A-B"
    },
    {
        "Room Type":  "Standard",
        "RowNumber":  50,
        "Student Name":  "Victor",
        "Student#":  "RL35",
        "LAN Port #":  "LAN Port 7-8",
        "Logical Seat Location":  "SL 10",
        "Actual Seat Location":  "Seat33",
        "Monitor Cable Port":  "C-D"
    },
    {
        "Room Type":  "Standard",
        "RowNumber":  51,
        "Student Name":  "Victor",
        "Student#":  "RL35",
        "LAN Port #":  "LAN Port 5-6",
        "Logical Seat Location":  "SL 10",
        "Actual Seat Location":  "Seat33",
        "Monitor Cable Port":  "A-B"
    },
    {
        "Room Type":  "Standard",
        "RowNumber":  52,
        "Student Name":  "Honey",
        "Student#":  "RL42",
        "LAN Port #":  "LAN Port 3-4",
        "Logical Seat Location":  "SL 10",
        "Actual Seat Location":  "Seat33",
        "Monitor Cable Port":  "C-D"
    }
]

Feel free to drop your feedback and inputs on this page. Till then, Happy Scripting!!!

Friday, 2 June 2017

Run PowerShell using Local System Account

You will very frequently come into a scenario where you will need to run some commands using local system account.
In my case, I was working on a windows service running using local system account. The windows service internally was calling PowerShell scripts.
I developed PowerShell scripts independently and they were absolutely working fine(running using my account which has elevated privileges), but when tested along with Windows Service, it failed due to various reasons like access and permission issues (as Windows service run scripts using local system account).
So to resolve this issue, I launched PowerShell with local system account, did required changes to code, performed complete testing and whoa I was all setup to go.
I thought to share all this information to you and save some of your time in troubleshooting.

Initial Setup

  1. Download PsExec from this link.
  2. It will ask you to download PSTools folder on your machine. Save it on your C:\ drive.

Open PowerShell with Local System Account

  1. Open Command Prompt
  2. Run below commands :
  3. Change directory to PSTools folder.
cd C:\PsTools
  1. Invoke PowerShell using PsExec.exe
.\PsExec.exe –i –s PowerShell.exe
  1. It will launch PowerShell. Validate current user
whoami


Open PowerShell ISE with Local System Account

  1. Open Command Prompt
  2. Run below commands :
  3. Change directory to PSTools folder.
cd C:\PsTools
  1. Invoke PowerShell using PsExec.exe
.\PsExec.exe –i –s PowerShell_Ise.exe
  1. It will launch PowerShell ISE. Validate current user
whoami






Whoa! You are good to go. Do all your work and testing required to be done using Local System Account.

Happy Coding!!!

Encode in C# and Decode in PowerShell

I had a requirement to read some data in C# windows application, store it in database  and then later use it in a PowerShell script.
It was also required that some data like ‘Keys’ should not be saved directly in database i.e. should not be visible directly to other database users.
One approach is to use complex encryption decryption logic when high data security is a requirement. Read more about Encryption and Decryption of Data here.
Another approach is to use Base64 encoding and decoding approach where very high data security is not a requirement. More about Base64 encoding here and decoding here.
I opted out for Base64 encoding approach as data was not getting shared with external sources where data security is very important and my applications and database were in the same network.
Here comes the coding blog ðŸ™‚ !!!
1. Encode in C# :-
   1: public string EncodeKey(string txtKey)
   2: {
   3:     byte[] passBytes = System.Text.Encoding.Unicode.GetBytes(txtKey);
   4:     string encodedKey = Convert.ToBase64String(passBytes);
   5:     return encodedKey;
   6: }
2. Decode in PowerShell :-
   1: Function ReadData
   2: {
   3:     $ConnectionString = "Data Source='';Initial Catalog='';Integrated Security=SSPI;"
   4:     $con = New-Object "System.Data.SqlClient.SQLConnection"
   5:     $con.ConnectionString = $ConnectionString
   6:     $con.Open()
   7:  
   8:     $sqlcmd = New-Object "System.Data.SqlClient.SqlCommand"
   9:     $sqlcmd.connection = $con
  10:  
  11:     $sqlcmd.CommandText = “Select Key from MyDemoTable where KeyIndex = 'FirstKey'
  12:     $ds = New-Object System.Data.DataSet
  13:     $da = New-Object System.Data.SqlClient.SqlDataAdapter($sqlcmd)
  14:     $return = $da.fill($ds)
  15:     
  16:     $encodedKeyFromDb = $ds.Tables[0].Rows[0][0]
  17:     $originalKey = [System.Text.Encoding]::Unicode.GetString([System.Convert]::FromBase64String($encodedKeyFromDb))
  18:  
  19:     ## Use the Key for Further Code ##
  20: }
Please note that Base64String encoding is very simple to reverse and do not involves any private or public key concept. In situations where data security is a requirement, Encryption Decryption approach should be followed.
Happy Coding!!!

Keeping PowerShell VisualBasic MessageBox in Focus

Hi All,
I faced issue where I was using Visual Basic MessageBox for showing messages to users, but the popups were getting hidden behind my tool window. I had to explicitly select them, bring them to front and take action on the popup.
I also tried using ‘System.Windows.Forms.MessageBox’ but it again resulted in a popup window that was hidden behind the tool.
Solution
To keep MessageBox in Focus and on top of all windows, you can use one of the enumerator values ‘ShowModal’ for MessageBox Style.
Examples are:-
1. For Error
[Microsoft.VisualBasic.Interaction]::MsgBox("Some error occurred.", 
"OKOnly,SystemModal,Critical", "Error")
2. For Warning
[Microsoft.VisualBasic.Interaction]::MsgBox("Please correct fields.", 
"OKOnly,SystemModal,Exclamation", "Warning")
3. Success Message
[Microsoft.VisualBasic.Interaction]::MsgBox("Processing Completed.", 
"OKOnly,SystemModal,Information", "Success")
A full working code would look like
[void] [System.Reflection.Assembly]::LoadWithPartialName("Microsoft.VisualBasic") 
[Microsoft.VisualBasic.Interaction]::MsgBox("Some error occurred.", "OKOnly,SystemModal,Critical", "Error")
[Microsoft.VisualBasic.Interaction]::MsgBox("Please correct fields.", "OKOnly,SystemModal,Exclamation", "Warning")
[Microsoft.VisualBasic.Interaction]::MsgBox("Processing Completed.", "OKOnly,SystemModal,Information", "Success")
There are various other combinations that are part of Message Box Style Enumerator
ApplicationModal, DefaultButton1, OkOnly, OkCancel, AbortRetryIgnore, YesNoCancel, YesNo, RetryCancel, Critical, Question, Exclamation,

Information, DefaultButton2, DefaultButton3, SystemModal, MsgBoxHelp, MsgBoxSetForeground, MsgBoxRight, MsgBoxRtlReading
Use the one which suits your need.

Happy Coding!!!

Invoking SQL Scripts using PowerShell

Invoke-Sqlcmd cmdlet is used to run sql commands. We will see in this blog on how we can run large SQL scripts file through PowerShell.
1. Read SQL Script in a variable.
   1: $sql = [Io.File]::ReadAllText('.\CreateMyDBTables.sql');
2. Import SQLPS module
   1: Import-Module 'SQLPS' -DisableNameChecking;
3. Invoke the script. Specify Server Instance, I have specified local instance in below example
   1: Invoke-Sqlcmd -Query $sql -ServerInstance '.';

The full PowerShell script will look like below:-
   1: $sql = [Io.File]::ReadAllText('.\InstallPortalDB.sql');
   2: Import-Module 'SQLPS' -DisableNameChecking;
   3: Invoke-Sqlcmd -Query $sql -ServerInstance '.';
Happy Scripting!!!

Request, Export and Import Certificate Using PowerShell

Hi,
I am targeting to create a personal certificate in this blog post, configure the certificate, export it in local machine and then import it in another remote machine.
1. Request new Certificate
   1: Set-Location 'Cert:\LocalMachine\My'
   2: $cert = Get-Certificate -Template Machine -Url ldap:///CN=contoso-PKI-CA -DnsName MyVM01.contoso.com -CertStoreLocation Cert:\LocalMachine\My
   3: $thumbprint = $cert.Certificate.Thumbprint
2. Manage Private Keys
   1: #manage private keys
   2: $cert = Get-ChildItem -Recurse "Cert:\LocalMachine\My\$thumbprint"
   3: $stub = "\Microsoft\Crypto\RSA\MachineKeys\"
   4: $programData = $Env:ProgramData
   5: $keypath = $programData + $stub
   6: $certHash = $cert.PrivateKey.CspKeyContainerInfo.UniqueKeyContainerName
   7: $certFullPath = $keypath + $certHash
   8: $certAcl = Get-Acl -Path $certFullPath
   9: $accessRule = New-Object System.Security.AccessControl.FileSystemAccessRule 'contoso\cloud_pack_setup', 'ReadData,FullControl', 'Allow'
  10: $certAcl.AddAccessRule($accessRule)
  11: Set-Acl $certFullPath $certAcl
3. Copy Certificate from one store to another store
   1: #Copy certificate from personal to intermediate certification authorities
   2: Export-Certificate -Type CERT -FilePath C:\OrchCert.cer -Cert "Cert:\LocalMachine\My\$thumbprint"
   3: Import-Certificate -CertStoreLocation Cert:\LocalMachine\CA -FilePath C:\OrchCert.cer
4. Export Certificate
   1: #export certificate (Orch)
   2: Export-Certificate -Type CERT -FilePath C:\OrchCert.cer -Cert "Cert:\LocalMachine\CA\$thumbprint"
5. Copy Certificate from local machine to remote Machine
   1: #copy certificate from Orch VM to Portal VM
   2: Set-Location C:\Windows\System32
   3: Copy-Item C:\OrchCert.cer -Destination \\CPPortal01\C$\OrchCert.cer -Force
6. Import Certificate in remote machine after it is copied
   1: #import certificate in portal vm (asp portal)
   2: Import-Certificate -CertStoreLocation Cert:\LocalMachine\CA -FilePath C:\OrchCert.cer
The above steps can be merged to create a whole PowerShell script that creates , exports and imports a certificate.

Update configuration files using PowerShell

Hi Readers,
We will see in this post on how we can edit web.config or other configuration files using PowerShell. There are already several posts available on internet which shows this functionality, however I faced difficulties in updating connection string in the configuration file as they are not direct. Below steps will help you in updating the connection strings as well.
Sample configuration file is shown below. Other sections of web.config file are not shown in this blog for simplicity.
   1: <configuration>
   2:   <connectionStrings>
   3:     <add name="TestDBEntities" connectionString="metadata=res://*/TestProject.csdl|res://*/TestProject.ssdl|res://*/TestProject.msl;provider=System.Data.SqlClient;provider connection string=&quot;data source=SQL01;initial catalog=TestDB;integrated security=True;MultipleActiveResultSets=True;App=EntityFramework&quot;" providerName="System.Data.EntityClient" />
   4:   </connectionStrings>
   5:   <appSettings>
   6:     <add key="SCVMMServerName" value="VMM01" />
   7:     <add key="SCVMMServerPort" value="8100" />
   8:   </appSettings>
   9: </configuration>
We will try to update ‘appSettings’ section and connectionStrings section of this configuration file.
1. Read configuration file in a XML variable
   1: $webConfig = 'C:\inetpub\wwwroot\VMMService\Web.config'
   2: $doc = (Get-Content $webConfig) -as [Xml]
2. Update ‘appSettings’ Section
   1: $obj = $doc.configuration.appSettings.add | where {$_.Key -eq 'SCVMMServerName'}
   2: $obj.value = 'CPVMM02'
3. Add new ‘appSetting’. You will need to create an XmlElement and append it as a child node to ‘appSettings’
$newAppSetting = $doc.CreateElement(“add”)
$doc.configuration.appSettings.AppendChild($newAppSetting)
$newAppSetting.SetAttribute(“key”,”SCVMMIPAdress”);
$newAppSetting.SetAttribute(“value”,”10.10.10.10″);
4. Update ‘connectionStrings’ section. Here is the tweak, you have to read the root element and then modify the connection string as shown below:-
   1: $root = $doc.get_DocumentElement();
   2: $newCon = $root.connectionStrings.add.connectionString.Replace('data source=SQL01','data source=SQL02');
   3: $root.connectionStrings.add.connectionString = $newCon
5. Save the configuration file
   1: $doc.Save($webConfig)
The combined code will look like below:-
$webConfig = 'C:\inetpub\wwwroot\TestService\Web.config'
$doc = (Get-Content $webConfig) -as [Xml]
$obj = $doc.configuration.appSettings.add | where {$_.Key -eq 'SCVMMServerName'}
$obj.value = 'CPVMM02'

$newAppSetting = $doc.CreateElement("add")
$doc.configuration.appSettings.AppendChild($newAppSetting)
$newAppSetting.SetAttribute("key","SCVMMIPAdress");
$newAppSetting.SetAttribute("value","10.10.10.10");
 
$root = $doc.get_DocumentElement();
$newCon = $root.connectionStrings.add.connectionString.Replace('data source=SQL01','data source=SQL02');
$root.connectionStrings.add.connectionString = $newCon
 
$doc.Save($webConfig)
The updated XML will contain modified values as shown below:-
<configuration>
 <connectionStrings>
 <add name="TestDBEntities" connectionString="metadata=res://*/TestProject.csdl|res://*/TestProject.ssdl|res://*/TestProject.msl;provider=System.Data.SqlClient;provider connection string=&quot;data source=SQL02;initial catalog=TestDB;integrated security=True;MultipleActiveResultSets=True;App=EntityFramework&quot;" providerName="System.Data.EntityClient" />
 </connectionStrings>
 <appSettings>
 <add key="SCVMMServerName" value="CPVMM02" />
 <add key="SCVMMServerPort" value="8100" />
 <add key="SCVMMIPAdress" value="10.10.10.10" />
 </appSettings>
</configuration>
Let me know if there are some sections of configuration file which you are finding difficult to update and I will add them here in this blog ðŸ™‚ Happy Scripting!!!