Quick and Dirty Scripting

SharePoint Trending with Google APIs

noreply@blogger.com (Brian Denicola) — Sun, 08 Nov 2009 02:38:00 +0000

At my day job, I manager a fairly large SharePoint deployment. We currently have about 450GB of content spread across 6-7 content databases. Needlessly to say, because SharePoint stores all of its information inside SQL Server database and Microsoft’s recommendation for no larger than 100GB content databases, trending our growth is very important to our team. For a while, we have been taking snapshots of the content database’s size, storing the information in Excel and generating Pivot table and charts. This worked fine but I wanted better. I’ve always liked the way Google’s Finance pages looked so I was excited to learn that you can use those same APIs in your own site. I think I’ve come up with a pretty neat solution by utilizing the SharePoint object model ,PowerShell, a little jQuery, Google’s APIs and an awesome course from EndUserSharepoint (Url - http://www.endusersharepoint.com, Twitter - @ESUP) . I’m always open for suggestions or improvements so after you read my posting, please let me know what you think.

Enough chit chat. Let’s get into how this works. To get this working for you, you’ll need three things:

A SharePoint List to store the size of the SharePoint databases
A PowerShell script to populate the list on a nightly basis
A Web Part page with a Content Editor Web part + jQuery and the SharePoint List to generate the graph

SharePoint List

This easiest thing to create. Its just a custom SharePoint list with the following columns.

Web Application – the name of the SharePoint web application. I typically rename the default Name field.
Database Name – the name of the of the Content Database.
Database Size – a number field that will store the size of the database. The one thing that I do not like about the number field is that it inserts commas into the number. I debated about making this a string field but I decided to just handle this with a little jQuery
Date – this will hold the date the database snapshot was taken. I give it a default value of today’s date and only select date, not date/time.

A couple other things to note about the list:

You will need to grant your SharePoint farm account contribute access to the list. This is because the PowerShell script will run as the Farm account. In a locked down environment like mine the only account that can access the databases is the Farm account.
To keep the list from growing too large, I setup an IRM policy to delete a list item 90 days after the item is created. This keeps my list nice and trim automatically.

PowerShell Script

The PowerShell script utilizes the SharePoint object model so it has to run on one of the SharePoint servers in your farm. I choose my Central Admin server but it can be any server in the farm. The idea of the script to is to loop through every Web Application and gather the size of each Content Database associated with it and then upload the information to the SharePoint list. Its pretty straight forward except for one issue with SharePoint Content Database names when working with PowerShell. This discussion thread goes into good detail on how to get around it - http://groups.google.com/group/microsoft.public.windows.powershell/browse_thread/thread/16c06d92b2385325.

   1: . ..\SharePointFunctions.ps1   2:     3: #Powershell had an issue with the SPContentDatabase assembly function Name and DiskSizeRequired. This is a work around   4: #See http://groups.google.com/group/microsoft.public.windows.powershell/browse_thread/thread/16c06d92b2385325 for more information    5: $nameMethod = [Microsoft.Sharepoint.Administration.SPContentDatabase].getMethod("get_Name")   6: $diskMethod = [Microsoft.Sharepoint.Administration.SPContentDatabase].getMethod("get_DiskSizeRequired")   7:     8: set-variable -option constant -name TrendingList -value "Trending-SPDatabase"   9: set-variable -option constant -name TrendingSite  -value "http://tis-collaboration.jpmchase.net/ti/shareddotnet"  10: set-variable -option constant -name WebApplication  -value "Web Application"   11: set-variable -option constant -name DatabaseName -value "Database Name"  12: set-variable -option constant -name DatabaseSize  -value "Database Size (MB)"  13:    14: $MB = 1024*1024  15:    16: #Function main - Where all of the fun is  17: function main() {  18:    19:     #Loop through all WebApplications  20:     get-SPWebApplications -name * | % {  21:           22:         #Don't need to display information on the Central Admin site  23:         if( -not $_.IsAdministrationWebApplication ) {  24:                           25:             $strWebApp = $_.Name.TrimStart()  26:                       27:             #Display information about the content database attached to this WebApp  28:             $ContentDatabaseCollection = $_.ContentDatabases  29:               30:             $ContentDatabaseCollection | % {  31:       32:                 $newlistitem  = @{}  33:                 $newlistitem[$WebApplication] = $strWebApp  34:                 $newlistitem[$DatabaseName] = $nameMethod.Invoke($_, "instance,public", $null, $null, $null)  35:                 $newlistitem[$DatabaseSize] = [math]::round( ($diskMethod.Invoke($_, "instance,public", $null, $null, $null) / $MB), 0)  36:                   37:                 add-toSpList -url $TrendingSite -list $TrendingList -entry $newlistitem   38:    39:             }  40:         }  41:     }  42:    43: }  44: main

The variables TrendingSite, TrendingList, WebApplication, DatabaseName, DatabaseSize will need to be updated for your environment.

The function get-SPWebApplication will return a list Web Application objects. You can limit the list to a specific Web Application by replacing “–name *” with “–name $name_of_web_application”

   1: function get-SPWebApplication( [string] $name )   2: {   3:     $WebServiceCollection = new-object microsoft.sharepoint.administration.SpWebServiceCollection( get-SPFarm )   4:     $WebServiceCollection | % { $WebApplications += $_.WebApplications }   5:        6:     return ( $webApplications | where { $_.Name.ToLower() -like "*"+$name.ToLower()+"*" } | select -Unique )   7: }   8:     9: function get-SPFarm()  10: {  11:     return [microsoft.sharepoint.administration.spfarm]::local  12: }

To add the information to the SharePoint List, we have to pass a hash table with name/value pairs to the function add-toSpList. We also have to pass the URL of the Site that hosts the list and the List name.

   1:     2: function add-toSpList ( [Object] $url, [string] $list, [HashTable] $entry)   3: {   4:     if( $url.GetType().Name -eq "String")   5:     {   6:         $site = new-object Microsoft.SharePoint.SPSite($url)   7:         $web = $site.OpenWeb()   8:     }else   9:     {  10:         $web = $url  11:     }  12:       13:     $splist = $web.Lists[$list]  14:     $newitem = $splist.items.Add()   15:    16:     $entry.Keys.GetEnumerator() | % {  17:         $newitem[$_] = $entry[$_]  18:     }  19:       20:     $newitem.update()   21:     $web.Dispose()  22:     $site.Dispose()   23: }

Now that we have to entire script, all you have to do is schedule it to run as often as you wish via the Windows Scheduler. Make sure you run the script as the SharePoint Farm Admin.

SharePoint WebPart Page + jQuery

Now the fun begins. I owe a ton of thanks to the EndUserSharepoint.com team as I used a lot of their work as a basis for what I’m trying to do. First, what you need to do is add a Web Part page to your site. On this page, add a Content Editor Web Part and the SharePoint List that you created earlier. For the list, I filtered the view to only display one database name. I also only selected he database name, size, and date columns. If I have more than one database (or web application) then I will need to create a page for each. The view should look something like this:

Next you have to get jQuery working with your site, and this means adding code to the Content Editor Web Part. I would suggest heading over to EndUserSharepoint.com (http://www.endusersharepoint.com/?s=jquery+for+everyone) for more information on jQuery, but basically jQuery is a collection of JavaScript functions and objects that make JavaScript coding a lot easier. You need to load the jQuery engine and thankfully Google hosts it for you. All you need to do is link to their URL in your code. This is what I done for this to work. Google also hosts the visualization APIs that we need to generate the graph. If you want to learn more about the available APIs and samples on how to use them, head over to http://code.google.com/apis/visualization/

The code for the Content Editor Web Part + jQuery is as follows: (PS: This has to be added via the CEWP’s source button, not the Rich Text Editor.)

   1: <script type="text/javascript" src="https://www.google.com/jsapi"></script>   2:     3: <script type="text/javascript">   4:     google.load('visualization', '1', {packages: ['annotatedtimeline']});   5:     function drawVisualization() {   6:       var dataList = $("td.ms-vb2 div");   7:       var dateList = $("td.ms-vb2 nobr");   8:       var dateArray = new Array();   9:    10:       $.each(dateList, function(i,e)  11:       {  12:         var y = $(e).text();  13:         dateArray[i] = y;  14:       });  15:    16:       var data = new google.visualization.DataTable();  17:       data.addColumn('date', 'Date');  18:       data.addColumn('number', 'Size');  19:       data.addRows(dataList.length);  20:         21:       $.each(dataList, function(i,e)  22:       {  23:          var x= $(e).text().replace(",","");  24:          data.setValue(i, 0, new Date(dateArray[i]) );  25:          data.setValue(i, 1, parseInt(x * 1048576) );  26:       });  27:         28:       var annotatedtimeline = new google.visualization.AnnotatedTimeLine(document.getElementById('trending'));  29:       annotatedtimeline.draw(data, {'displayAnnotations': true});  30:     }  31:       32:     google.setOnLoadCallback(drawVisualization);  33:    34: </script>  35: <div id="trending" style="width: 100%; height: 400px;"></div>

Well there you go. Once you have everything in place you should have a page the looks similar to this:

I hope that everyone has enjoyed this post and I always look forward to any feedback.

Time zone Clocks with SharePoint using jQuery

noreply@blogger.com (Brian Denicola) — Fri, 16 Oct 2009 04:31:00 +0000

I work with a lot people who are spread across the global so we are always trying to find out what is their local time. While Google has a great query that will convert time for you, I wanted to see if I could create a SharePoint page that will list the times in all the zones that we have people on the team. I found a great blog post on ‘Path to Sharepoint’ (http://pathtosharepoint.wordpress.com/2009/01/09/world-clock/) did exactly what I wanted. The one catch was that he used a lot of JavaScript to convert the calculated text column to HTML. I wondered if jQuery could make it easier. This is what I came up with . . .

  1: <script type="text/javascript" src="http://www.google.com/jsapi"></script>
  2: <script type="text/javascript">
  3:     //load jQuery
  4:     google.load("jquery", "1.3.2");
  5: </script>
  6:
  7: <script type="text/javascript">
  8: $(function() {
  9:     $("td.ms-stylebody:contains('DIV')").each(function(){
 10:         $(this).html($(this).text());
 11:     });
 12: });
 13: </script>

There are a couple requirements for this to work.

The list view was set to boxed.
The calculated column had this equation:

="<DIV id='calchtml'><embed src='http://www.clocklink.com/clocks/5001-blue.swf?TimeZone="&TimeZone&"'
width='250' height='70' wmode='transparent' type='application/x-shockwave-flash'></DIV>"

If you follow the instructions at ‘Path to SharePoint’ and use this jQuery you should get something like this:

Powershell Script to Create Sharepoint Web Application

noreply@blogger.com (Brian Denicola) — Thu, 10 Apr 2008 02:13:00 +0000

Microsoft Sharepoint 2007 has a very nice web-based UI to create Web Applications, but there is one catch. The UI creates the backend database on the fly. In a lot of companies, this is the role of a DBA, and only they are allowed to create a new database. In a way this makes sense due to the fact that the DBAs must maintain the intergity of the database server. Microsoft has provided a switch within the stsadm command to create a new Web Application with an existing database, but I thought I could come up with a way to do it using Powersell.

What I wanted to do was to supply a script with an XML configuration that would create 1-N Web Applications with 1-N Sites. The XML looks like the following:

<Sharepoint>
<WebApplication name="Test Site #1" hostheader="http://portal.example.net">
<AppPoolName>WebApp-Portal</AppPoolName>
<AppPoolUser>Domain\myServiceAccount</AppPoolUser>
<AppPoolPass>test1235</AppPoolPass>
<Port>80</Port>
<DatabaseServer>DBS-SERVER-NAME\MOSS</DatabaseServer>
<DatabaseName>WSS_Content_Potal</DatabaseName>
<RootDirectory>d:\inetpub\wwwroot\wss\portal</RootDirectory>
<Sites>
<Site Path="/">
<Title>Root Site</Title>
<Description>Root Site</Description>
<Type>STS#1</Type>
<AdminAccount>DOMAIN\Administrator</AdminAccount>
<AdminName>Administrator1</AdminName>
<AdminEmail>root@jexample.net</AdminEmail>
</Site>
</Sites>
</WebApplication>
<WebApplication></WebApplication>
</Sharepoint>

The first thing that needs to be done is load the Sharepoint .NET assembly and attach to the local farm

[void][System.Reflection.Assembly]::LoadWithPartialName("Microsoft.SharePoint")
$farm = [microsoft.sharepoint.administration.spfarm]::local

Then load the XML configuration file and loop through all Web Applications.

function main()
{

$moss = "Sharepoint"
$cfg = [xml](gc $cfgFile)

if( $? -eq $false ) {
Write-Host "Could not cleanly parase XML file. Exiting . . ."
return $false
}

Write-Host "Found Sharepoint Farm on Local Host . . "

Write-Host "Using $cfgFile file . . ."

$cfg.$moss.WebApplication | % {

createWebApp( $_ )

}
}

main

Surprisingly to create new Web Application, all you need to do is create a Powershell object of type SPWebApplicationBuilder, assign the proper variables, create and then provision. The command can take up to 10 minutes to complete so be patient.

function createWebApp( [object] $cfg )
{
$webAppBuilder = $nul
$webAppBuilder = new-object _
microsoft.sharepoint.administration.SPWebApplicationBuilder($farm)

$secureString = ConvertTo-SecureString $cfg.AppPoolPass -asPlainText -force

$webAppBuilder.Port = $cfg.port
$webAppBuilder.ApplicationPoolId = $cfg.AppPoolName
$webAppBuilder.ApplicationPoolUsername = $cfg.AppPoolUser
$webAppBuilder.ApplicationPoolPassword = $secureString

$webAppBuilder.HostHeader = $cfg.hostheader
$webAppBuilder.ServerComment = $cfg.name
$webAppBuilder.DatabaseServer = $cfg.DatabaseServer
$webAppBuilder.DatabaseName = $cfg.DatabaseName
$webAppBuilder.RootDirectory = $cfg.RootDirectory

if( $cfg.AllowAnonymous.ToString().ToLower() -eq "true" ) {
$webAppBuilder.AllowAnonymousAccess = $true
}

Write-Host "Will now Provision this Web Application."
Write-Host "This may take up to 10 minutes. . ."
$webApp = $webAppBuilder.Create()
$webApp.Provision()

Finally, loop through all Sites that are listed in the XML and add them to the Web Application.

$cfg.Sites.Site | % {
$title = $_.SiteTitle.ToString()
$path = $_.Path.ToString()

rite-Host "Will now Provision Site - $title ($path). "
Write-Host "This may take up to 10 minutes. . ."

$webApp.Sites.Add( $_.Path,
$_.Title,
$_.Description,
1033,
$_.Type,
$_.AdminAccount,
$_.AdminName,
$_.AdminEmail)
}

See easy as 1, 2, 3 . . .

Powershell Scriptblocks

noreply@blogger.com (Brian Denicola) — Mon, 11 Feb 2008 05:08:00 +0000

One of the interesting features that is, I believe, little know in Powershell is the scriptblock. What is a scriptblock? The way that I define it is a function or block of code that is assigned to a variable. Why do you need to know about this? I've found it most useful when using Powershell to create a GUI. You have to assign an action to an element on the GUI (like a Button). But the action can not be a function. It must be a script block. You can define the block as a variable or define it within the {} of the Add_Click method of the System.Windows.Forms.Button object.

Now I'm probably completely wrong with this, and if I am I would love to learn more about it. So please let me know! I want to cover how to create GUIs in Powershell in couple weeks so I will be building off this blog post.

So how do you create a Script block. Its easy as assigning a variable to a anything within {}.

A simple case would be something like

PS C:\temp> $x = { Write-Host "Hello, World!" }

PS C:\temp> $x.GetType()

IsPublic IsSerial Name BaseType
-------- -------- ---- --------
True False ScriptBlock System.Object

If you just call the variable, it will return the block definition

PS C:\temp> $x
Write-Host "Hello, World!"

To invoke the scriptblock, you need to call the Inovke() method of the script

PS C:\temp> $x.Invoke()
Hello, World!

Within the scriptblock, you can do whatever you want to do. It can be multiple lines, you can even call functions from within the block. For example:

PS C:\temp> function y () {
>> Write-Host "Within function y"
>> }
>>
PS C:\temp> y
Within function y
PS C:\temp> $x = {
>> Write-Host "Within scriptblock x"
>> y
>> }
>>
PS C:\temp> $x.Invoke()
Within scriptblock x
Within function y

Here are all of the methods that Scriptblocks provides within Powershell

PS C:\temp> $x | Get-Member

TypeName: System.Management.Automation.ScriptBlock

Name MemberType Definition
---- ---------- ----------
Equals Method System.Boolean Equals(Object obj)
GetHashCode Method System.Int32 GetHashCode()
GetType Method System.Type GetType()
get_IsFilter Method System.Boolean get_IsFilter()
Invoke Method System.Collections.ObjectModel.Collection`1[[System.Management.Automation.PSObject, Syst...
InvokeReturnAsIs Method System.Object InvokeReturnAsIs(Params Object[] args)
set_IsFilter Method System.Void set_IsFilter(Boolean value)
ToString Method System.String ToString()
IsFilter Property System.Boolean IsFilter {get;set;}

I hope this gives you an introduction into scriptblocks. Next time, hopefully, we can cover the TCL/TK-like feature of Powershell - its GUI.

Q&D Ping Function for Powershell

noreply@blogger.com (Brian Denicola) — Wed, 16 Jan 2008 21:29:00 +0000

By default, Powershell does not have any built in functionality to send an ICMP ping to see if a remote computer is alive on and on the network. But never fear because Powershell is based off .NET. This means it is very easy to implement a Ping function that you can add to a Powershell library file (along with other functions that I have blogged about previously). The function is pretty straight forward. All that you really new to do is to create two .NET objects - Ping and PingReply. My ping function only looks to see if a success was returned by ICMP. I am not looking for round trip time or to handle non-successful.


function Ping (  [string] $strComputer )
{
  $timeout=120;
  trap { continue; }
  
  $ping = new-object System.Net.NetworkInformation.Ping 
  $reply = new-object System.Net.NetworkInformation.PingReply

  $reply = $ping.Send($strComputer, $timeout);
  if( $reply.Status -eq "Success"  ) 
  {
     return $true;
  }
  return $false;

}

Again that's it. The trap command will just ignore any errors that are throw by the $ping.Send command. If anyone has questions than you can always email me at brian@bjd145.org.

Powershell and XML Configuration Files

noreply@blogger.com (Brian Denicola) — Mon, 07 Jan 2008 04:54:00 +0000

One of the things that I almost always create when I am developing a script is a configuration file. Utilizing XML for a configuration file format is very easy with Powershell. The first thing that you need to do is to layout your XML file. I typically don't create a DTD or verify the XML schema because well, only my scripts will be using the XML configuration file. Not completely within spec, but I've learn to live with it. The XML does need to be well formed with proper closing brackets, root node, etc in order for Powershell to read the XML but that is the only restriction.

The most common XML file that I create includes a list of servers and some attribute about those servers. A simple example that of a test application that is made up of three web servers and a database server. The way that I would create the XML would be as follows:

<test_app_servers>
    <dbs server="vm-dbs1" dbsname="eut"/>
    <web dir="d$\inetpub\wwwroot">
        <server>vm-test1</server>
        <server>vm-test2</server>
        <server>vm-test3</server>
    </web></test_app_servers>

Now the important part, how does Powershell read the XML. The code is very straight forward. You use the get-content command and cast the file as XML. The command is:

$cfgFile = ".\test_app_config.xml"
$root = "test_app_servers"
$cfg = [xml] ( gc $cfgFile )

Now once you have the configuration saved in the variable $cfg then how do you use it? Powershell stores the information in a tree format so it is easy to get at. Some samples of how to get at the data include:

$strDbsName = $cfg.$root.dbs.dbsname
$strDbsServer = $cfg.$root.dbs.server
$strWebDir = $cfg.$root.web.dir
$cfg.$root.web.server | % {
        $strWebServer = $_
        "Content of \\$strWebServer\$strWebDir"
        dir \\$strWebServer\$strWebDir
}

And that is all you should need to know in order to utilize XML as a configuration file with Powershell

Update to VMware's Powershell Toolkit

noreply@blogger.com (Brian Denicola) — Thu, 27 Dec 2007 17:22:00 +0000

Just wanted to throw this out there for anyone using VMware's Powershell toolkit. They released a new CTP on 12/26. The file version is 68764. They've added a bunch of new commands such as Get-Inventory, New/Get-OSCustomizationSpec, Add-VMHost, Move-Datacenter, Move-Cluster, Find-EntityViews, etc . .

I need to find some time to play with these new commands. I do know that the base Get-VIServer and Get-VMHost do work with ESX 3.5 which is nice to see as well.

Until I have more scripts to share. . .

How to Find all Domain Computers Using Powershel

noreply@blogger.com (Brian Denicola) — Mon, 03 Dec 2007 03:53:00 +0000

I'm asked a lot to find out something from all the servers in my Active Directory domain. Like "Tell me the free space on all servers in the domain" or "List all of the error events from Microsoft's Event Viewer for servers in the domain." These two questions can be answered very easily using WMI and Powershell,namely the get-WMIobject function. But how to get all of the server names? Using VBScript, this can be accomplished by using ADSI. But the code is hard to follow and I never really got a hang of using the ADSI calls. But why uses that when there is a great tool made already. dsquery. dsquery is tool written by Microsoft to query Active Directory (http://technet2.microsoft.com/WindowsServer/en/Library/46ba1426-43fd-4985-b429-cd53d3046f011033.mspx). Using this and combined with Powershell is Quick and Dirty but as always gets the job done.

So how do you use it? First the basic . . .

dsquery computer "ou=Domain Controllers,DC=example,DC=com" will find all servers in the Organizational Unit Domain Controllers. dsquery has a option 'server' which will do the same thing (dsquery servers)

The return result is . . .
"CN=DC03,OU=Domain Controllers,DC=example,DC=com"
"CN=DC02,OU=Domain Controllers,DC=example,DC=com"
"CN=DC01,OU=Domain Controllers,DC=example,DC=com"

To return all computers, just do ....
dsquery computer -limit 0 - The limit 0 tells dsquery not to limit the number of systems it returns. I believe the default is 100.

dsquery can also be used to find servers in the domain . . . .
dsquery computer -name "*WEB*" will return
"CN=DRWEB01,OU=Web,OU=Disaster Recovery,OU=Machines,OU=Shared Applications,DC=example,DC=com"
"CN=DEV-WEB01,OU=Development,OU=Machines,OU=Shared Applications,DC=example,DC=com"
"CN=STG-WEB01,OU=Stage,OU=Machines,OU=Shared Applications,DC=example,DC=com"
"CN=STG-WEB02,OU=Stage,OU=Machines,OU=Shared Applications,DC=example,DC=com"
"CN=PRD-WEB01,OU=Production,OU=Machines,OU=Shared Applications,DC=example,DC=com"

So can this be used with Powershell. As long as dsquery is located in the path where powershell can call it from, just write a function such as this

function GetDomainServers {
return @( dsquery computer -limit 0 -o rdn | % { $_.Split("`"")[1] } )
}
The -o rdn means to just return the server name. The server name will be in quotes which is why the return values are piped into Split("`""). This will split the server name into 3 parts. We just want the second value. This is combined to form an array which is returned.

In other words,
dsquery computer -limit 0 -o rdn returns "PRD-WEB01".
The powershell command Split("`"") will split the value on ".
This returns
"
PRD-WEB01
"

To call this function, its as easy as $servers = GetDomainServers.

Well that's it. As always, I would love to hear feed back from anyone who finds these posts interesting and helpful.

Restoring A Database with transaction logs

noreply@blogger.com (Brian Denicola) — Wed, 28 Nov 2007 18:44:00 +0000

I've been often asked to restore a database to another server and have all transactions logs applied to it up to a certain point. Using SQL Server 2005, you could setup Transaction Log shipping and it works very well, or you could copy the files over manually then apply them manually using SQL Server Enterprise Studio. Well I'm lazy so I came up with this simple script to do the work for me . . .

This script will ask for the database name to be restored as well as the directory where the transaction logs , drop any existing database of that name, restore the database and its transaction logs. It will not setup your database backups (full or transaction) nor will it replicate them over. I do have scripts for this but a tool like SyncBackSE works just as well. Oh this is a Windows batch script that utilizes osql which comes with SQL Server 2005.

@echo off
set BAKFILE=%1
set DBSNAME=%2
set LOGDIR=%3
IF NOT DEFINED BAKFILE (GOTO HELP)
IF NOT DEFINED DBSNAME (GOTO HELP)
IF NOT DEFINED LOGDIR (GOTO HELP)
echo Using the following variables to restore the database
echo Database Backup file - %BAKFILE%
echo Database Name - %DBSNAME%
echo Trans Log Directory - %LOGDIR%
choice /M "Do you want to continue"
IF ERRORLEVEL 2 ( GOTO END)
set DATADIR=d:\DATA
set SQL=%TMP%\restore.sql
set OUT=.\restore_results.txt
IF EXIST %SQL% ( del %SQL% )
IF NOT EXIST %BAKFILE% (
echo %BAKFILE does not exist. Must exit.
GOTO END
)
echo USE [MASTER] > %SQL%
echo GO >> %SQL%
echo ALTER DATABASE %DBS% SET SINGLE_USER WITH ROLLBACK_IMMEDIATE >> %SQL%
echo GO >> %SQL%
echo DROP DATABASE %DBS% >> %SQL%
echo GO >> %SQL%
echo RESTORE DATABASE [%DBSNAME%] > %SQL%
echo FROM DISK = '%BAKFILE%' >> %SQL%
echo WITH NORECOVERY >> %SQL%
echo GO >> %SQL%
for /f "tokens=* delims= " %%a in ('dir /b /a-d /od %LOGDIR%\*.trn') do (
echo RESTORE LOG [%DBSNAME%] >> %SQL%
echo FROM DISK = '%LOGDIR%\%%a' >> %SQL%
echo WITH NORECOVERY >> %SQL%
echo GO >> %SQL%
set LAST=%%a
)
echo RESTORE LOG [%DBSNAME%] >> %SQL%
echo FROM DISK = '%LOGDIR%\%LAST%' >> %SQL%
echo WITH RECOVERY >> %SQL%
echo GO >> %SQL%
osql -E -w 5000 -i %SQL% -o %OUT%
GOTO END
:HELP
echo.
echo Usage: restore_dbs.bat [BAKFILE] [DBSNAME] [LOG DIR]
echo.
echo.
echo BAKFILE - The full path to a recent backup file of the database
echo DBSNAME - Name of the new database
echo LOG DIR - The full path to a directory that contains all the recent tanslogs
echo *The LOG DIR should not contain any transaction logs older than BAKFILE
echo.
echo Eg: restore.bat d:\Backups\SampleDBS_backup_200705222201.bak SampleDBS d:\Temp
echo.
:END
IF EXIST %SQL% ( del %SQL% )

VMware's Very Cool Get-Stat Command

noreply@blogger.com (Brian Denicola) — Mon, 26 Nov 2007 14:11:00 +0000

Today I want to briefly show you the power of the VMware's Get-State command. With it you can grab statistics on any VI entity - ESX Host, Virtual Machine, or Resource Pool. To use it, all you need to do is supply the entity you want to monitor and which statistics do you want to grab (common, CPU, memory, disk, or network). So let's get some stats on an ESX host.

$vc = Get-VIServer "vc1.example.com"

$esx = Get-VIHost "esx1.example.com"

Get-Stat $esx -Memory -maxsamples 3 -realtime

MetricId             Timestamp                                     Value Unit
---------            ----------                                    ----- ----
mem.vmmemctl.average 11/26/2007 1:52:2...                              0 KB
mem.vmmemctl.average 11/26/2007 1:52:0...                              0 KB
mem.vmmemctl.average 11/26/2007 1:51:4...                              0 KB
Value       : 21.49
Timestamp   : 11/26/2007 1:52:20 PM
MetricId    : mem.usage.average
Unit        : %
Description : Memory usage as percentage of total configured or available memory
Entity      : VMware.VimAutomation.Client20.VMHostImpl
Value       : 21.49
Timestamp   : 11/26/2007 1:52:00 PM
MetricId    : mem.usage.average
Unit        : %
Description : Memory usage as percentage of total configured or available memory
Entity      : VMware.VimAutomation.Client20.VMHostImpl
Value       : 21.49
Timestamp   : 11/26/2007 1:51:40 PM
MetricId    : mem.usage.average
Unit        : %
Description : Memory usage as percentage of total configured or available memory
Entity      : VMware.VimAutomation.Client20.VMHostImpl
mem.active.average   11/26/2007 1:52:2...                          78988 KB
mem.active.average   11/26/2007 1:52:0...                          78988 KB
mem.active.average   11/26/2007 1:51:4...                          78988 KB
mem.granted.average 11/26/2007 1:52:2...                        2097504 KB
mem.granted.average 11/26/2007 1:52:0...                        2097504 KB
mem.granted.average 11/26/2007 1:51:4...                        2097504 KB
Get-Stat $esx -common -maxsamples 1 -realtime
MetricId             Timestamp                                     Value Unit
---------            ----------                                    ----- ----
cpu.usage.average    11/26/2007 2:05:4...                           4.86 %
Value       : 297
Timestamp   : 11/26/2007 2:05:40 PM
MetricId    : cpu.usagemhz.average
Unit        : MHz
Description : CPU usage in MHz over the collected interval. For hosts this can be represented on a per Virtual Machine
              basis as a stacked graph
Entity      : VMware.VimAutomation.Client20.VMHostImpl
mem.usage.average    11/26/2007 2:05:4...                          21.48 %
Value       : 105
Timestamp   : 11/26/2007 2:05:40 PM
MetricId    : disk.usage.average
Unit        : KBps
Description : Aggregated storage performance statistics. For hosts this can be represented on a per Virtual Machine bas
              is as a stacked graph
Entity      : VMware.VimAutomation.Client20.VMHostImpl
And finally,

$stats = Get-Stat $esx -common -maxsamples 1 -realtime

$stats. GetType()

IsPublic IsSerial Name                                     BaseType
-------- -------- ----                                     --------
True     True     Object[]                                 System.Array
Since the object returned to you is an array,this allows you to store and manipulate the return results - like

$stats[2]

MetricId             Timestamp                                     Value Unit
---------            ----------                                    ----- ----
mem.usage.average    11/26/2007 2:07:0...                          21.48 %

or $stats[2].Value

21.48

These values can then be piped into a table or graph to display the results.

Next week, I want to go back to something basic in Powershell - XML configuration. Powershell makes parsing XML documents very easy and I want to show how you can use this to create XML configuration files.

First post using Windows Live Writer

noreply@blogger.com (Brian Denicola) — Wed, 21 Nov 2007 23:06:00 +0000

This is a posting using Microsoft Live Writer which means this will be a short post.

Just want to say Happy Thanksgiving!!!!

Brian

Creating a VM from a Template using Powershell

noreply@blogger.com (Brian Denicola) — Wed, 14 Nov 2007 17:04:00 +0000

Yesterday, I posted that I was having issues creating a new VM from a template while using VMware's Powershell commands. It must have been the weather or something because today, it seems to be working without any issues. The following is how you can create a new VM from a template using Powershell. I did notice one interesting thing that I want to ask VMware about which I will mention at the end of this entry.

PS C:\Temp> $vc = Get-VIServer "vc.example.com"
There were one or more problems with the server certificate:

* The certificate's CN name does not match the passed value.

PS C:\Temp> $t = Get-Template -Server $vc
PS C:\Temp> $t[0]

Name Id
---- --
ESX3-WIN2K3SP2-ST... VirtualMachine-vm-23637

PS C:\Temp> $esx = Get-VMHost -Name "esx1.example.com"
PS C:\Temp> $esx

Name Id
---- --
esx1.example.com HostSystem-host-7523

PS C:\Temp> New-VM -Host $esx -Template $t[0]

cmdlet New-VM at command pipeline position 1
Supply values for the following parameters:
Name: vm-1

At this point, a Powershell progress bar appears and the progress of the deployment can be tracked. The shell's focus is not returned until after the deployment is complete. The interesting thing is, that I can CTRL-C and cancel this Powershell command but the template continues to be deployed from within VirtualCenter. I am not sure how VMware will handle this situation but I would expect the deployment to be canceled once I cancel the Powershell command. I would love to hear any one else's thoughts on this matter as well.

Creating a Virtual Machine with Powershell

noreply@blogger.com (Brian Denicola) — Tue, 13 Nov 2007 14:38:00 +0000

Sorry for the delay in posting this week. I had out-of-town guests in all weekend so everything has been turned upside down. I would like to continue from last week and show how easy it is to create a new virtual machine with Powershell. Again you'll need the VMware Powershell management Snap-in before you have access to these commands.

To Create a Virtual Machine, you need to do the following:
1. Connect to your Virtual Center server -

$vc = Get-VIServer -Server "vc.example.com"

2. Connect to the ESX Host where you want to create the VM -

$esx = Get-VMHost -Server $vc -Name "esx.example.com"

3. Create the virtual machine -

$vm = New-VM $esx
This will create a VM with a 4GB hard drive and 256MB of RAM.
The network will be the first Network Label sorted alphabetically

4. If you want, you can specify options while you are creating the VM or you can use the Set-VM to modify the VM's specifications.

Set-VM $vm -MemoryMB 2048 -NumCPU 4 -Network "Local Lan"
Proceed with update of the following parameters of virtual machine with name 'blah'?
New MemoryMB: 2048MB
New NumCPU: 4
New Networks: LAN-1,
[Y] Yes [A] Yes to All [N] No [L] No to All [S] Suspend [?] Help (default is "Y"): y

I tried to create a new VM from a template with the following steps, but I continually got an error while doing so. I guess this just goes to show that VMware is still developing this code and it has some ways to go.

1. Get all templates that VC knows about

$t = Get-Templates -Server $vc

2. Create new virtual machine from the first template

$vm = New-VM -Host $y -Template $t[0]
cmdlet New-VM at command pipeline position 1
Supply values for the following parameters:
Name: blah2
New-VM : The operation for the entity vm-3179 failed with the following message: "A specified parameter was not correct

If someone could show me what I am doing wrong, that would be great.

PowerShell and VMware Managment

noreply@blogger.com (Brian Denicola) — Tue, 06 Nov 2007 04:53:00 +0000

I know that I just blogged yesterday about sending emails with PowerShell, BASH, and VBScript, but I got a chance today to install VMware's PSSnapin for Powershell. From the looks of this beta, its going to be a very powerful product that almost anyone can write scripts for. What an example.

I wrote a script in Perl using VMware's Perl SDK to display any VM Snapshot that a supplied VirtualCenter knows about. The code to do this in Perl is as follows:

use strict;
use warnings;
use Getopt::Long;
use VMware::VIRuntime;
use Term::ReadKey;

my %opts = (server => undef, username => undef,);

GetOptions (\%opts,"server=s", "username=s");

if( !defined ($opts{server} && $opts{username}) ) {
   help();
   exit (1);
}

ReadMode('noecho');
print "Enter $opts{username}\'s password: ";
my $password = ReadLine(0);
chomp $password;
ReadMode('normal');

# login
my $url = "https://" . $opts{server}. "/sdk/vimService";
Vim::login(service_url => $url, user_name => $opts{username}, password => $password);

# get VirtualMachine views for all powered on VM's
my $vm_views = Vim::find_entity_views(view_type => 'VirtualMachine');

# snapshot each VM
my $i;
foreach (@$vm_views) {
 if( defined $_->snapshot ) {
  print "-------------------------------------------------------------\n";
  print $_->name . " has at least one snapshot defined  . . . \n";
  foreach $i (0 .. $#{$_->snapshot->rootSnapshotList} ) {
     print "\t" . $_->snapshot->rootSnapshotList->[$i]->name . " @ " . $_->snapshot->rootSnapshotList->[$i]->createTime . "\n";
  }
  print "-------------------------------------------------------------\n";
 }
}

# logout
Vim::logout();

sub help {
   my $help_text = <<'END';  USAGE:    query_snapshot.pl --server  --user 

Example:
   query_snapshot.pl --server vc1 --user administrator

END
   print $help_text;
}

Now using Powershell and the VMware Snapin:


 Add-PSSnapin VMware.VimAutomation.Core
 Get-VIServer vc1.example.com
 Get-VM | % { $_.Name; Get-Snapshots $_ }

That's it. Everything is provided for you like Authentication, handling XML returned by VirutalCenter's Web Services, etc . . . The Perl SDK is very powerful as well, but you have to do a lot more coding to get things done. With the PowerShell snapin, the script is all of three lines. Now the snapin is only in beta testing right now so things can change in the future.

Sending email via a Script

noreply@blogger.com (Brian Denicola) — Mon, 05 Nov 2007 04:00:00 +0000

One of the most basic tasks that any system administrator needs to accomplish while writing scripts is to send e-mail. Typically this is to send results of script itself or of some event.

The easiest way to send email is if you are using Linux. Almost all Linux servers have the mail command installed by default. This command can be used by a BASH script to send your mail. The problem with the mail program is that it doesn't send attachments. To send attachments, you'll need mutt. Mutt can be installed via apt-get or yum or via tar balls, on all Linux systems. To send an email via Mutt, you simply need to have the following line in your script


mutt -s $subject -c $ccpt_list -a $attachfile $rcpt_list < $email

The variables defined include:
1. A subject ($subject)
2. A receipt list ($rcpt_list). This is defined in BASH as rcpt_list=someone@company.com.
3. The body of the email is stored in the variable/file $email. $email is a file on the server that is piped into the Mutt command. It is defined as email=/tmp/email.tmp echo Hello World >> $email
4. OPTIONAL: An attach file. The variable $attachefile holds the path location of the file to attache
5. OPTIONAL: An CC list of email address.
--

Over in the Microsoft world, there isn't a command line mail program built into any Windows server, but VBScript is. All that you need to send email via VBScript is the following:

Const SMTP_SERVER = "SMTP mail server address"
Const SMTP_PORT = 25

Set objMessage = CreateObject("CDO.Message")
objMessage.Sender = sSender 'Email address of sender
objMessage.To = sRecipients 'Comma separate list of people who will receive the email
objMessage.Subject = sSubject 'The subject of the email
objMessage.Textbody = sBody 'The body of the email.

'This is optional.  sAttachment is the fully qualified path to the file to attach
objMessage.AddAttachment sAttachment

objMessage.Configuration.Fields.Item ("http://schemas.microsoft.com/cdo/configuration/sendusing") = 2
objMessage.Configuration.Fields.Item ("http://schemas.microsoft.com/cdo/configuration/smtpserver") = SMTP_SERVER
objMessage.Configuration.Fields.Item ("http://schemas.microsoft.com/cdo/configuration/smtpserverport") = SMTP_PORT
objMessage.Configuration.Fields.Update

objMessage.send

---

With Powershell just being released from Microsoft, you can write a function to send email and include that function in all scripts that need to send email. The function could look something like this:

function send-email($s,$b,$to) {
  $from = "send@company.com"; //Who will be sending the email
  $domain  = "smtp.company.com"; //SMTP server to send the message

  $mail = new-object System.Net.Mail.MailMessage;

  for($i=0; $i -lt $to.Length; $i++) {
     $mail.To.Add($to[$i]);
  }
  $mail.From = new-object System.Net.Mail.MailAddress($from);
  $mail.Subject = $s;
  $mail.Body = $b;

  $smtp = new-object System.Net.Mail.SmtpClient($domain);
  $smtp.Send($mail);
}

To call this function, you just need to do the following:

$SUBJECT="This is the subject";
$EMAILADDR=@("person1@company.com","person2@company.com");
$BODY="Hello World";
send-email -s $SUBJECT -b $BODY -to $EMAILADDR

Sending email is pretty easy and I know that there are a lot of places on the Web that already so you how to send email. Hopefully, though we can build off this to build much more complex scripts. I received an email from VMware concerning their Powershell interface to ESX. I am planning on next week to show how to write some simple scripts to manage an ESX build out with Powershell and we'll be using the send-email functions.

Well this ends my first real blog post. I hope someone out there finds these scripts/functions useful and I will write more next week.

First Post!

noreply@blogger.com (Brian Denicola) — Mon, 29 Oct 2007 16:41:00 +0000

Hi. My name is Brian Denicola and this is the first post for what I hope will be a weekly blog that focuses on scripting for system administrators. I have working in the IT world for the past 10 years and have been automating tasks for most of them. This blog will focus on getting things done using any language available. But I'll mainly focus on Windows batch scripting, Bash scripting, VBscript, Perl, and Powershell. Every once in a while I might throw in a how to script for ASP.NET or SQL. I I won't be going into too much theory of the language, rather I hope to show how to get things done in Windows, Linux, or even VMware ESX. And I hope this will become a site where others can post how they do things as well.

Well that is all for now. Next time I will be show you how to send e-mail via powershell. I'll start simple and go from there.