What makes a good Software Test engineer?

“A good test engineer has a ‘test to break’ attitude, an ability to take the point of view of the customer, a strong desire for quality, and an attention to detail. Tact and diplomacy are useful in maintaining a cooperative relationship with developers, and an ability to communicate with both technical (developers) and non-technical (customers, management) people is useful. Previous software development experience can be helpful as it provides a deeper understanding of the software development process, gives the tester an appreciation for the developers’ point of view, and reduce the learning curve in automated test tool programming. Judgment skills are needed to assess high-risk or critical areas of an application on which to focus testing efforts when time is limited.”

In my opinion, more emphasis should be placed on the statement italicized above. Taking it a step further, I would say a developer with previous development knowledge of the product under test is even more valuable. Obviously domain knowledge is what increases this value. From previous experiences with organizations, the development skills of the tester are often largely ignored.

“Writing code? That’s what our developers do, not our testers.”

If we are not placing a greater emphasis on the development skills of the tester, we are missing opportunities to fully test the product. Testers with a development background are more familiar with the developer’s perspective. They know the tricks developers do when writing code, they know their shortcuts and tendencies. These testers can also provide developers with insight into defensive coding, adding hooks to make automation easier, securing the product from attacks, ect.

I would argue the inverse is true as well, that having previous testing experience of the product would add value to the transition of a tester into development. A developer with a testing background should improve the testability and security of the product, because they, too, understand the importance and impact of test.

To me, test and development should be shared responsibilities. If a developer is out sick, on vacation, or the team needs a resource, a tester should be able to fill in, developing product code. If a tester is needed for similar reasons, a developer should be able to switch gears and fulfill that need as well. Neither should be more exciting nor glamorous. The organization should respect both disciplines equally. Testing is the last line of defense. 

The software engineer should have two personae, constructive and destructive. Perhaps this is similar to Dr. Jekyll and Mr. Hyde, where we replace the “evil” in Mr. Hyde with “destructive product testing.”

I recently put together some notes for a presentation on cross site scripting, or XSS for short. I have decided to share some of the information, because I believe keeping a few thoughts in mind as we develop and test will go a long way when it comes to preventing attacks.

Cross Site Scripting – What is it?

Cross site scripting occurs when a web application gathers malicious data entered from a user, with or without negative intent. XSS can be achieved by exploiting locations in source code where users are able to input data, and the proper preventative measures have not been implemented to format and validate the inputted data. In other situations, session cookies and other sensitive data can be accessed by injecting malicious data and ultimately hijacking or impersonating a user’s account.

There are three main types of XSS according to Wikipedia, so I won’t bother rephrasing. Visit http://en.wikipedia.org/wiki/Cross-site_scripting for more information.

The most common technologies and languages used for XSS are JavaScript, VBScript, ActiveX, HTML, or Flash’s ActionScript. Not only do we need to protect our server code, but we need to think about our client code as well.

Prevention of Cross Site Scripting

The vast of majority of XSS attacks can be prevented by identifying the user input locations within the web application and ensuring the source code handling these has proper measures in place. From a developer’s perspective, this means ensuring all data inputted from a user is properly encoded to remove HTML and script markup to be replaced with text that all browsers can process.

A simple example in C# is to use the HttpServerUtility.HtmlEncode method to convert all HTML markup characters into their text equivalent. For example, if a user were to supply the input for a textbox with the following, “This is my <b>bolded text</b>.” The end result of the HtmlEncode method would result in the following, “This is my lt;bolded textgt;.” This is important because it removes HTML markup, which could be malicious. For example, “This is my text. <script>alert(‘This is an attack’);</script>.” This example is passing a JavaScript alert to open a modal popup on the screen to display to the user.

In addition to HTML inputted data encoded on the server, encoding data on the client can be equally important. JavaScript HTML elements can have two attributes, InnerHTML and InnerText. InnerText will render text, not HTML, so it is the safe option. InnerHTML can be used to inject an XSS attack because it can render user inputted HTML, including script. Ensure InnerHTML has the necessary string formatting to protect against this vulnerability.

Cookies are another vulnerability to XSS attacks. If any part of the website issues cookies and an XSS access point is discovered, it is now possible to steal cookies and private information from the application’s users. If the cookie can be accessed, so can the information with it. Users can be impersonated, and site credibility will be lost.

Encrypted web sites (SSL, HTTPS) are at risk just like their public counterparts. SSL sites appear to be protected, but it is possible to execute the same XSS attacks, they just happen over an encrypted connection.

To protect our web applications, we need to be aware of the XSS vulnerabilities common to attackers and place defensive measures to ensure user confidentiality and confidence. Without becoming an expert on XSS and security, it is possible to develop safe, reliable applications by understanding XSS and the vulnerabilities exposed by our applications.

What to Look for in Source Code

Execute a simple search in source code looking for certain keywords is a good starting point. Many of the XSS bugs I have seen reported could have been prevented with the simple measures. Ensure HTML input is properly encoded on the server using HtmlEncode. Ensure HTML input is properly formatted on the client using string.Format and InnerText.

In source code and wherever, look for the following vulnerabilities.

• InnerHTML
  • InnerText is not supported by all browsers but the two can be found together. Look for where the strings are originating and if they are properly formatted/encoded.
• SetInnerText()
• JavaScript’s Eval()
  • Don’t be Eval – http://24ways.org/2005/dont-be-eval
• Assigning of strings to page titles, control titles, ect.
  • Sometimes we take request object data and immediately process it and render it on the client.
  • Check the URL parameters passed in.
• The Request object
  • Request.Params
  • Request.Forms
  • Request.QueryString
• Using HtmlTextWriter or any variation
  • RenderBeginTag()
  • AddAttribute()
  • RenderEndTag()
  • HtmlWriter.Write()
• Cookies
  • Where are we using them and how are we handling them

Searching for these keywords within source will be a decent starting point for discovering XSS vulnerabilities

Microsoft Anti-Cross Site Scripting Library V3.0 Beta

Feel free to use a Microsoft API designed for XSS prevention within your code.

http://www.microsoft.com/downloads/details.aspx?FamilyId=051ee83c-5ccf-48ed-8463-02f56a6bfc09&displaylang=en

Closing Notes – because this topic can go on forever

There are tools to help assess if your site is vulnerable. Search for them online. Whether or not you think you need third party APIs to help you write defensive code is entirely up to you. You can always write the code yourself.

Cross-site scripting (XSS) can be damaging to a company’s credibility and can cause myriad undesirable effects for individual users. XSS is preventable. Familiarizing oneself with the smells of XSS is a valuable tool to posses as a developer and a tester. At a minimum, educate your developers and testers on the target hot-spots mentioned within this post.

References and Resources

http://en.wikipedia.org/wiki/Cross-site_scripting

http://www.cgisecurity.com/xss-faq.html

http://ha.ckers.org/xss.html

http://www.owasp.org/index.php/Cross_site_scripting

To create a new file that is 100 MB, we have this command.

A quick and easy way to generate files of various sizes.

Back to my original investigation, Live Mesh, while it says it supports files of up to 2 GB, the actual uploading of movies (avi, wmv, mpg) greater than 50 MB appear to timeout. I was hoping to share video with family and friends. Hopefully there will be a resolution in the near future.

Read more on FSUTIL.

FSUTIL.exe (Win XP/2003 server)

FSUTIL on TechNet

As a tester, I am constantly creating new environments, installing our product, taking snapshots, ect. Doing this by hand is a waste of time in my opinion. Doing this with a script makes life easier and frees up my time.

Typically, I prefer to create is to create one VM and to have other VMs inherit from that base image. My second VM (the first child to inherit from the base), I will install daily builds of our product. I will then create a third child, which I use a sandbox. I can easily delete it and recreate as needed without having to install our product over again. I have not consistently found success with using snapshots, so I prefer to use differencing disks. The examples below use differencing disks.

This first code block shows how I am using the PowerShell management library to simply the creation of a base VM. There is nothing magic about it. It’s pretty strait-forward. I create a VM, give it a name, set the CPU count, memory size, network adapters, hard drive, and DVD drive.

$server = "my-dev-server"
$vmName = "TestParentVM"
$vmVirtualSwitch = "My Virtual Network"

# create a new virtual machine
$vm = New-VM $vmName -server $server

# set cpu count
Set-VMCPUCount $vm 2 -server $server

# set memory size
Set-VMMemory $vm 2 -memory 4GB -server $server

# add a legacy network adapter
Add-VMNIC -vm $vm -VirtualSwitch $vmVirtualSwitch -legacy

# add a default VMBus (non-legacy) network adapter
Add-VMNIC -vm $vm -VirtualSwitch $vmVirtualSwitch

# add the hard drive to the VM
Add-VMNewHardDisk –vm $vm -controllerID 0 -lun 0 -vhdpath "$(get-VHDdefaultPath)\$vmName.vhd" -size 20GB

# add the DVD with bootup ISO
Add-VMDRIVE –vm $vm 1 1 -server $server -DVD 

Write-Host "Operation complete."

The next code block will create a differencing disk pointing to the parent created in the previous example. I create the usual, VM, CPU count, ect. What makes this a differencing disk is how the VHD is created by specifying the parent. I do some cleanup in this routine by deleting any previously created VMs or VHDs, and I loop through the creation a failure preventative.

$server = "my-dev-server"
$vmName = "TestChildVM"
$vmParent = "TestParentVM"
$vmVirtualSwitch = "My Virtual Network"

#region functions - feel free to move to a separate file

Function DeleteVhd
{
    Param ($pathToVhd)
    # delete the disk that was added if it exists
    if (test-path -path $pathToVhd)
    {
        Remove-Item -Path $pathToVhd -Force
    }
}

#endregion

# delete the VHD if it already exists
DeleteVhd "$(get-VHDdefaultPath)\$vmName.vhd"

# delete the VM if it already exists
Remove-VM -vm $vmName -server $server

# create a new virtual machine
$vm = New-VM $vmName -server $server

# get the vm
$vm = Get-VM $vmName

# set cpu count
Set-VMCPUCount $vm 2 -server $server

# set memory size
Set-VMMemory $vm 2 -memory 4GB -server $server

# get VM Nics available
$vmNics = Get-VMNic -server $server -vmbus -legacy
$vmNicSwitchList = New-Object System.Collections.ArrayList

# get VM Nic Switch available for each VM Nic and add it to our list
foreach ($vmNic in $vmNics)
{
    $vmSwitchElementName = (Get-VMNicSwitch $vmNic).ElementName
    if ($vmNicSwitchList.Contains($vmSwitchElementName) -ne $true)
    {
        $vmNicSwitchList.Add($vmSwitchElementName)
    }
}

# Use the first available VM Nic Switch - this assumes
# I do not create more than on per each environment.
if ($vmNicSwitchList.Count -gt 0)
{
    $vmVirtualSwitch = $vmNicSwitchList[0]
}

# add a legacy network adapter
Add-VMNIC -vm $vm -VirtualSwitch $vmVirtualSwitch -legacy

# add a default VMBus (non-legacy) network adapter
Add-VMNIC -vm $vm -VirtualSwitch $vmVirtualSwitch

# add the hard drive to the VM
$parent = "$(get-VHDdefaultPath)\$vmParent.vhd"
$vhdPath = "$(get-VHDdefaultPath)\$vmName.vhd"
Add-VMNewHardDisk –vm $vm -controllerID 0 -lun 0 -vhdpath $vhdPath -parent $parent

# get the VM to see if a disk is attached to it, if it is not, then it failed
$disks = Get-VMDisk -vm $vm
$count = 1

# if the disk was not successfully added, try and add it again
while ($disks -eq $null -and $count -le 5)
{
    Write-Host "RETRY $count - The disk was not properly added. Attempting to retry."

    # delete the disk that was added
    DeleteVhd "$(get-VHDdefaultPath)\$vmName.vhd"

    Add-VMNewHardDisk –vm $vm -controllerID 0 -lun 0 -vhdpath $vhdPath -parent $parent

    $disks = Get-VMDisk -vm $vm
    $count++
}

if ($disks -eq $null)
{
    Write-Host "Operation failed to create this virtual machine. Contact an admin."
}
else
{
    Write-Host "Operation completed successfully. Attempting to start the VM."
    Start-VM -vm $vm -wait
}

I have found that the VM creation is not consistently successful on the first pass, so I have added the while loop. Since adding that, I have not had any issues.

I hope these help. They should be easy to change if you prefer snapshots versus differencing disks. Thanks to James O’Neill for creating this library. It makes administering Hyper-V much easier. It is available via CodePlex, where the latest release, updates, and forum support can be found.

This past weekend, I decided to upgrade my work laptop from Vista Enterprise to Windows 7 Ultimate Beta. I was unable to upgrade from Vista Enterprise to 7 Ultimate because that action is not supported. I then tried to upgrade from Enterprise to 7 Enterprise, but I was unable to upgrade from a staged build to a non-staged build. I decided to repave entirely, choosing Windows 7 Ultimate Beta, and I tried to do all of this on a Friday afternoon, two hours before leaving.

I installed Windows 7 Beta Checked Build (x86) – DVD (English), last updated on January 14, 2009,  from my MSDN subscription. I did not install an internal build. Instead, I decided to go with what is publicly available via MSDN.

Why did I go with a Checked Build? Because I failed to read the details describing it. What it cost me in time and frustration, I gain in valuable learning experiences. So it was not a total loss. I received a number of buggy issues with the checked build, mainly the inability for Windows Update to properly download all of my necessary updates, including my video drivers and smart card reader, where the latter enables me to remotely access my work domain.

I gave up and decided to download and install the Windows 7 Beta – DVD (English). Everything works great. I was able to get online, download all of my updates, and connect to my work domain. No issues found so far, and no annoying assertion error popups.

My favorite Windows 7 feature so far (non-UI related) is that PowerShell 2.0 is installed by default. PowerShell 2.0 provides more functionality for remoting, among other enhancements.

There are so many versions of Linux, so which one should I choose?

I feel like a kid in a candy store when it comes to selecting one, or maybe two operating systems. I began with Ubuntu, essentially, picking up from the last chapter where I grew tired and put down the book that is Linux. Ubuntu worked well, or at least, it did not leave a bad taste in my mouth. Last time I installed Ubuntu, I was using Virtual PC 2004, and it worked successfully.

Installing Ubuntu 8.10 in Hyper-V was incredibly easy. A full installation required 3-4 screens of user interaction. After downloading the ISO image to my server, I fired it up in Hyper-V. First, select a language. Second, choose to try, install, check for defects, test memory, ect. I chose to “install” versus “try” a live version. Third, user input screen, answer some install configuration questions such as language, time zone, keyboard layout, disk space partition, and login information. Finally, after installing, I am prompted for my login and password. Total time was less than thirty minutes, and perhaps it could have been faster, but I was multitasking. After installing Ubuntu and configuring FireFox to work with my network, I was testing my web application.

Still in awe of how easy the installation was, I decided to try out other flavors of Linux. Like Microbrews, Linux distros seem to be a dime-a-dozen. I found some online articles to help me narrow down what distros other seems to like, and so I arrived at the following.

The distros I downloaded and installed (or tried to install) as a Hyper-V Virtual Machine.

• Ubuntu 8.10
• Fedora 10
• Linux Mint 6
• Mandriva Linux One 2009 (Gnome and KDE) – tried both
• Open SuSe 11.1 KDE
• SimplyMEPIS 7.0

In Hyper-V, installing Fedora, Linux Mint, and Open Suse were strait-forward and successful on my first run, just like Ubuntu. I had trouble with Mandriva and SimplyMEPIS. Both of them would hang as they tried to load the ISO image, so I gave up. At this point, I have four working Linux VMs enabling me to test my web applications, so the incentive to get Mandriva and SimplyMEPIS working just is not there right now. I do believe Mandriva and SimplyMEPIS will work with Hyper-V. I must have some configurations incorrect.

One thing I thought was funny about SimplyMEPIS is their website poses a question as part of their advertising, “Why SimplyMEPIS?” They respond with, “SimplyMEPIS just works!” It is sad that to have to advertise that “it just works,” because to me, that implies the inverse was the rule rather than the exception with previous versions, or with the technology – in this case, Linux.

I have fought Linux installs in the past, getting my onboard sound card to work, locating other device drivers, and even upgrading browsers. When I see, “it simply works,” I am still skeptical. It is unfortunate in my case with Hyper-V, that SimplyMEPIS did not work within the time limits I cared to allow for each distro. I will give it and Mandriva a fair shake by trying other configurations to get them working.

From my experiences, virtual machines are a great place to start getting familiar with operating systems. Having access to Hyper-V certainly made this experience successful and enjoyable. VMware and Virtual PC are two other virtualization software applications I have used as well for similar situations. VMware’s support for Linux operation systems is great. I have been able to install Fedora Core and Ubuntu in Virtual PC, but not without swearing at my screen.

Faster, easier, safer and more reliable. Check it out for yourself.

Strait from the mouth of marketing.

Internet Explorer 8 RC1 is much faster than previous versions of Internet Explorer, so you’ll notice that difference right away.  But you’ll also notice some cool new features that will make your surfing experience faster and easier—not mention a whole lot safer.

• Highlight a street address in your Hotmail, Yahoo Mail, or other web mail account, right-click on the blue button that appears, and hover over “Map with Live Search.”  Presto!  You’ll see the map with the location right there—no more copying and pasting street addresses from web mail to a mapping site.  Plus, you can choose what mapping service you want to use.
• Go to the search box in the top right, and type a search item—see how the enhanced Instant Search Box is more helpful, providing real-time search suggestions, including images, from your chosen search provider.
• Click on a new tab, and see the options that are presented there, including “InPrivate Browsing.”  Click InPrivate Browsing and watch what happens—you are now in a browser session that is leaving no trail behind, so research gift suggestions for your significant other to your heart’s content without worrying about who might pick up the crumbs after you.

Harder to see are the many security enhancements that will help keep you protected against the ever-evolving online threats—things like protection from cross-site scripting attacks that no other browser offers.

Get-Help Remove-Item -Full

As I was retrieving information on “Remove-Item,” this is what I came across. There is more detail beyond what I am showing, but pay attention to what is bolded.

NAME
    Remove-Item

SYNOPSIS
    Deletes the specified items.

SYNTAX
    Remove-Item [-path] <string[]> [-recurse] [-force] [-include <string[]>] [-
    exclude <string[]>] [-filter <string>] [-credential <PSCredential>] [-whatI
    f] [-confirm] [<CommonParameters>]

    Remove-Item [-literalPath] <string[]> [-recurse] [-force] [-include <string
    []>] [-exclude <string[]>] [-filter <string>] [-credential <PSCredential>]
    [-whatIf] [-confirm] [<CommonParameters>]

DETAILED DESCRIPTION
    The Remove-Item cmdlet deletes one or more items. Because it is supported b
    y many providers, it can delete many different types of items, including fi
    les, directories, registry keys, variables, aliases, certificates, and func
    tions.

PARAMETERS
    -path <string[]>
        Specifies a path to the items being removed. Wildcards are permitted. T
        he parameter name (“-Path”) is optional.

        Required?                    true
        Position?                    1
        Default value                N/A – The path must be specified
        Accept pipeline input?       true (ByValue, ByPropertyName)
        Accept wildcard characters?  true

    -recurse <SwitchParameter>
        Deletes the items in the specified locations and in all child items of
        the locations.

        The Recurse parameter in this cmdlet does not work properly.

        Required?                    false
        Position?                    named
        Default value                False
        Accept pipeline input?       false
        Accept wildcard characters?  false

 

“Hmmm… Am I feeling lucky? Ahhh, what the hell, let’s do it.”

Remove-Item "C:\" -Recurse

For example, every night I clean my build environment, sync up the latest code, and rebuild all products within my domain. The total time it takes to clean, sync, and build is over two hours, depending on hardware and connection speed. This has been reduced from over four hours. Now before I do all of this, I need to check my environment to ensure I have all the latest tools and versions of those tools, such as Visual Studio, SQL Server, windows updates, service packs, ect. This is just part of our build environment. Typically, we run this check until we receive a positive response, “all is well.”

Rather than spend my time running these commands, I automate them, and run them around the clock using Task Scheduler, a BAT file, and PowerShell.

I create one scheduled task that runs once per hour, indefinitely. My scheduled task runs a BAT file that calls my PowerShell script. Within my PowerShell script, I create a Hashtable of commands to run and at which hour to run them. Each hour as the script is executed, I get the current hour and see if any tasks are available to run. For example, each night, sometime during the 2 AM hour, I start my full build. I realize that running once per hour limits the number of commands I can run, but for now, this imperfect solution meets my needs.

 1 # Local variables
 2 $productRoot = "c:\Projects\dev"
 3 $currentHour = (Get-Date).Hour
 4 $list = New-Object System.Collections.Hashtable
 5 
 6 # Define tasks to run each hour
 7 #     Hour 0 = 12:00-12:59 AM
 8 #     Hour 1 = 1:00-1:59 AM
 9 #     Hour 20 = 8:00-8:59 PM
10 $list.Add(20,"environmentCheck dev");
11 $list.Add(21,"environmentCheck test");
12 $list.Add(22,"environmentCheck system");
13 $list.Add(23,"projects clean sync");
14 $list.Add(2,"projects build full");
15 
16 # if a task is defined for this hour, execute it
17 if($list.Contains($currentHour))
18 {
19     # Get the command to execute for this hour
20     $buildCommand = $list[$currentHour]
21 
22     # Set the project bat path
23     $pathToBat = $productRoot + "\bin\project.bat"
24 
25     # Set the cmd.exe's arguments
26     $arguments = "/C cd /d " + $productRoot + " & " + $pathToBat + " & " + $buildCommand
27 
28     # Start up the DOS prompt, execute the commands
29     $process = New-Object -TypeName System.Diagnostics.Process
30     $process.StartInfo.FileName = "cmd.exe"
31     $process.StartInfo.Arguments = $arguments
32     $process.StartInfo.UseShellExecute = $false
33     $process.Start()
34     $process.WaitForExit()
35 } 

Every morning, I expect to have fresh builds. My build system will email me with the results. However, adding email capabilities to the above script with PowerShell is doable. The above commands execute on my dev server, but executing them on remote servers is also doable, even with PowerShell 1.0 and something like the following.

1 $toServer = "my-remote-server"
2 ([WMICLASS]"\\$toServer\ROOT\CIMV2:win32_process").Create($command)

If my environment is not current and my builds are not working, I can lose productivity during normal business hours. Using a simple scheduled task and PowerShell helps me administer my server and keeps me from repeating the same tedious steps on a daily basis.

How else does PowerShell help me adhere to DRY? It enables me to administer my Hyper-V machines, configure websites, GAC and un-GAC assemblies, and much more.

I use XML/XPath/XSL less frequently today than with previous positions, but every now and then, when chiseling away at a slab of XML, having an XPath tool can make my life much easier. I always relied on Interactive XPath Expression Builder 4.0, but larger XML files hindered performance. In an effort to find a good XPath editor, I have installed several I have found on the web, used them, and uninstalled all of them.

Ideally, I am looking for a tool that is light and free. By light I mean, I am not looking to install a development studio, nor something that attempts to integrate with all of my existing applications, nor something that requires a reboot when installing or uninstalling, nor something that cannot fully uninstall itself. I plan on using this tool as often as I have to deal with XML/XPath/XSL these days, which is once every few months. Usually the time it takes me to find a tool, install it, author what I need, and uninstall it, I could have written the XPath by hand.

As I searched online for tools this time, I stumbled across SketchPath. While hesitant to install another XML app to only end up in my graveyard of XML apps, I gave this one a try. It is free, so I figured, why not.

“YES! F-bomb YEAH!”

After installing, I loaded up my XML, clicked on an attribute within a node, and sure enough, the XPath needed to locate that element was displayed. It was that simple. After a few more node selections, I had the XPath I needed for my app. In and out like a robbery.

“SketchPath is a free XPath Editor and XML analysis and testing tool supporting XPath 1.0 and XPath 2.0. It provides an integrated graphical environment for viewing XML files, developing and testing XPath expressions against them and managing the expressions in file libraries.”

Thanks Phil! This was exactly what I needed. I will not be uninstalling.