Tech in the 603, The Granite State Hacker

Keeping in the Code

At the end of the day, the business solution is always the most important part of the equation, but it’s not the only part.  While I’m working on a solution, I’m also looking at tools, scaffolding, and framework.  This is especially true if others are going to be working on the project, and that accounts for nearly every non-trivial project.

How easy is it to set up?  How easy is it to work with?  Do the expressions make sense?  Can I hand it off to my least experienced teammate, get them to pick this up, and expect reasonable results?  (For that matter, can I hand it off to my most experienced teammate and expect them to respect the design decisions I made? )

Keeping my head in the code is critical.  Loosing touch with tools means shooting in the dark on the above questions.  It doesn’t matter what their experience is, if you ask someone to push a tack into a corkboard, hand them the wrong tools for the job, they won’t be able to push the thumbtack into the corkboard… or you’ll nuke your budget paying for tools that are overpowered for the job.  (But that thumbtack will be SO IN THERE!)

In any case, in most projects, after the architecture & technical designs have been sorted out, frameworks, built, automations put in place, I’ll take on the coding, too.

Of course, I’ve said this before…  if you can really simplify the work, what’s to stop you from taking the extra step and automating it?   I’m always eyeing code, especially “formulaic”, repetititive stuff, looking for opportunities to simplify, abstract, and/or automate.

Tech in the 603, The Granite State Hacker

Custom Guid’s

Caught a question from Stacy Draper, @StacyDraper this morning about custom guids, to make them more recognizable.

It reminded me of a post I saw recently about Facebook using hex characters to make IPv6 addresses more recognizable.

Here’s what I was thinking… create a guid that has an embedded word.

For example, the following code creates a Guid that always starts with FACEB00C:

using System;
usingSystem.Collections.Generic;
using System.Linq;
usingSystem.Text;
 
namespaceCustomGuidTest
{
    class Program
    {
        static void Main(string[] args)
        {
            GuidcustomGuid = GenerateCustomGuid();
            Console.WriteLine(customGuid.ToString(“B”));
            Console.ReadKey();
        }
 
        static Guid GenerateCustomGuid()
        {
            Guidresult;
 
            //0xFACEB00C
            //backwards, but required this to achieve desired result
            //Likely due to some standard with respect to bit order.
            byte[] custom = new byte[] { 0x0C, 0xB0, 0xCE, 0xFA  };

            byte[] random = Guid.NewGuid().ToByteArray();
            byte[] final = new byte[16];
            for(int idx = 0; idx <16; idx++)
            {
                if(idx >= custom.Length)
                {
                    final[idx] = random[idx];
                }
                else
                {
                    final[idx] = custom[idx];
                }
            }
            result = newGuid(final);
            returnresult;
        }
    }
}

Example output:
{faceb00c-7828-4e99-8cf5-280e33202670}

Tech in the 603, The Granite State Hacker

GSSPUG Hub (Free App) for Windows Phone Now Available

My “artisan portfolio” of Windows Phone apps just DOUBLED in size!  Yes, I’ve now successfully published my second Windows Phone app.  🙂

The Granite State SharePoint Users Group Hub is a somewhat minimal app, but if you’re a member of the group, it’s got some useful features.   My favorites are being able to get info about the next meeting, (both in the app, and as a live tile) and being able to RSVP through EventBright.

The direct link to find it in the Marketplace on your Windows Phone is this.

Regarding the name…  GSSPUG?  Ya, I know… it’s not quite as intuitive as NHSPUG…    

If you’re from New Hampshire, you know you search for “Granite State” any time you’re looking for something local…  and if you don’t know that, it probably is just as well you don’t find it.  😉

One other nice thing is that the content is largely driven from the group’s web site, which, of course, is a SharePoint site.   The app does require a network connection, but it can be updated without having to go through the week-long process of publishing an update. 

Like Jimmy Sudoku, the app uses your phone’s system wide theme colors.

Essentially this is what ends up in the Hub app.

And it appears like so:

Tech in the 603, The Granite State Hacker

Windows Authentication fails due to machine name alias issue

Another tidbit I ran into recently that I hit once long ago, and don’t want to forget it again:

I have a VM who’s name I changed.  Trying to develop a WCF webservice for an SOA initiative, I ran into a completely inexplicable authentication failure when I tried to authenticate with my own local user on the machine.

Turns out the quick & easy fix (identified in this post from MS  http://support.microsoft.com/kb/926642 ) ended up pointing me to the solution of:

 adding

REG_MULTI_SZ key “BackConnectionHostNames” to my registry at

HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Control\Lsa\MSV1_0 . 

One alias per line…

I added a couple lines for various aliases that I use for the machine, enabling me to authenticate as {alias}\{userName}, locally, on the machine.

Tech in the 603, The Granite State Hacker

Url for WSDL coming back with wrong Host name in WCF service

Here’s something I don’t want to forget…  working on a WCF Service…

Using TLS, the address of the wsdl didn’t match what was in the SSL certificate.  (the server certificate had the FQDN “friendly name” (host.domain.com), and I was just getting back the “host” name. 

Naturally, this was causing problems. 

There was lots of guidance about setting the host name using some old vbs script that turned out to be a red-herring for IIS 7, anyway.

The quick & simple solution was to add in the behavior for the service:

<system.serviceModel>

    <behaviors>
      <serviceBehaviors>
        <behavior>
          <useRequestHeadersForMetadataAddress />
          <!– To avoid disclosing metadata information,

          set the value below to false and remove the metadata

               endpoint above before deployment –>
          <serviceMetadata httpGetEnabled=false httpsGetEnabled=true  />
          <!– To receive exception details in faults for debugging purposes,

          set the value below to true.  Set to false before deployment

          to avoid disclosing exception information –>
          <serviceDebug includeExceptionDetailInFaults=True />
        behavior>
      serviceBehaviors>
    behaviors>

  system.serviceModel>
configuration>

Tech in the 603, The Granite State Hacker

Workstation Virtualization

I’ve been having fun (ya, really… fun!) with MS VPC 2007 SP1 lately.

I’ve put some VM’s on a portable USB disk. USB2 isn’t the best connection, but it’s workable. It does a few things… adds a spindle to the system config, offloading that overhead from the main disk.

An external disk also means the VM is “portable”. I can launch the VM on my laptop or any other host system (like my home system) without any significant difficulty. Even better, by having the VPC config files on the host, rather than on the external disk, you can tune VM settings (like memory and network connectivity) for optimal conditions on the host system.

One other nice time-saver is differencing disks. You can create a “base” virtual hard disk, and create VHD’s that are deltas of the base… by doing this, you can create a primary configuration, and then create several machines that inherit that basic config. It came in very handy for a recent product evaluation… I just created a base VM roughly according to what the client expects to host the system on, and then created VMs based on that for each product I wanted to evaluate.

Another nice feature is virtual assist hardware. At first, I didn’t know my ThinkPad had it, but it turns out to be a BIOS setting. Flip that on, do a cold boot, and VM performance is visibly better. I knew of some of the other features from past experience with MS VPC 2005, but the hardware acceleration is new to me. (Ironically, my newer home desktop, a 64-bit monster with huge RAM doesn’t support the hardware assist… performance isn’t a problem, there, tho. )

One more trick: enable the Undo disk option… It put another layer of protection on your VM, allowing you snap a line on your VM at a point in time that you can back-out to. The cool part about this is that the Undo disk is created as a temporary file on the host system, (typically on the primary system drive). This distributes load across the spindles even more, which further improves run-time performance. The downside: when it comes time to commit the undo, it can take a while.

I still love the idea, going forward, of putting client dev environments on a config like this… Not only does it create a nice level of separation between client system configurations, but when you get your hands on better hardware, migration is not an issue.

Tech in the 603, The Granite State Hacker

“ETL”ing Source Code

The past couple weeks, I’ve been between projects, which has gotten me involved in a number of “odd jobs”. An interesting pattern that I’m seeing in them is querying and joining, and updating data from very traditionally “unlikely” sources… especially code.

SQL databases are very involved, but I find myself querying system views of the schema itself, rather than its contents. In fact, I’m doing so much of this, that I’m finding myself building skeleton databases… no data, just schema, stored procs, and supporting structures.

I’m also pulling and updating metadata from the likes of SharePoint sites, SSRS RDL files, SSIS packages… and most recently, CLR objects that were serialized and persisted to a file. Rather than outputting in the form of reports, in some cases, I’m outputting in the form of more source code.

I’ve already blogged a bit about pulling SharePoint lists into ADO.NET DataSet’s. I’ll post about some of the other fun stuff I’ve been hacking at soon.

I think the interesting part is how relatively easy it’s becoming to write code to “ETL” source code.

Tech in the 603, The Granite State Hacker

Reading SharePoint Lists into an ADO.Net DataTable

[Feb 18, 2009: I’ve posted an update to show the newer technique suggested below by Kirk Evans, also compensating for some column naming issues.]

The other day, I needed to write some code that processed data from a SharePoint list. The list was hosted on a remote MOSS 2007 server.

Given more time, I’d have gone digging for an ADO.NET adapter, but I found some code that helped. Unfortunately, the code I found didn’t quite seem to work for my needs. Out of the box, the code missed several columns for no apparent reason.

Here’s my tweak to the solution:

(The ListWebService points to a web service like http://SiteHost/SiteParent/Site/_vti_bin/lists.asmx?WSDL )

private data.DataTable GetDataTableFromWSS(string listName)

{

ListWebService.Lists lists = new ListWebService.Lists();

lists.UseDefaultCredentials = true;

lists.Proxy = null;

//you have to pass the List Name here

XmlNode ListCollectionNode = lists.GetListCollection();

XmlElement List = (XmlElement)ListCollectionNode.SelectSingleNode(String.Format(“wss:List[@Title='{0}’]”, listName), NameSpaceMgr);

if (List == null)

{

throw new ArgumentException(String.Format(“The list ‘{0}’ could not be found in the site ‘{1}'”, listName, lists.Url));

}

string TechListName = List.GetAttribute(“Name”);

data.DataTable result = new data.DataTable(“list”);

XmlNode ListInfoNode = lists.GetList(TechListName);

System.Text.StringBuilder fieldRefs = new System.Text.StringBuilder();

System.Collections.Hashtable DisplayNames = new System.Collections.Hashtable();

foreach (XmlElement Field in ListInfoNode.SelectNodes(“wss:Fields/wss:Field”, NameSpaceMgr))

{

string FieldName = Field.GetAttribute(“Name”);

string FieldDisplayName = Field.GetAttribute(“DisplayName”);

if (result.Columns.Contains(FieldDisplayName))

{

FieldDisplayName = FieldDisplayName + ” (“ + FieldName + “)”;

}

result.Columns.Add(FieldDisplayName, TypeFromField(Field));

fieldRefs.AppendFormat(“”, FieldName);

DisplayNames.Add(FieldDisplayName, FieldName);

}

result.Columns.Add(“XmlElement”, typeof(XmlElement));

XmlElement fields = ListInfoNode.OwnerDocument.CreateElement(“ViewFields”);

fields.InnerXml = fieldRefs.ToString();

XmlNode ItemsNode = lists.GetListItems(TechListName, null, null, fields, “10000”, null, null);

// Lookup fields always start with the numeric ID, then ;# and then the string representation.

// We are normally only interested in the name, so we strip the ID.

System.Text.RegularExpressions.Regex CheckLookup = new System.Text.RegularExpressions.Regex(“^\\d+;#”);

foreach (XmlElement Item in ItemsNode.SelectNodes(“rs:data/z:row”, NameSpaceMgr))

{

data.DataRow newRow = result.NewRow();

foreach (data.DataColumn col in result.Columns)

{

if (Item.HasAttribute(“ows_” + (string)DisplayNames[col.ColumnName]))

{

string val = Item.GetAttribute(“ows_” + (string)DisplayNames[col.ColumnName]);

if (CheckLookup.IsMatch((string)val))

{

string valString = val as String;

val = valString.Substring(valString.IndexOf(“#”) + 1);

}

// Assigning a string to a field that expects numbers or

// datetime values will implicitly convert them

newRow[col] = val;

}

}

newRow[“XmlElement”] = Item;

result.Rows.Add(newRow);

}

return result;

}

// The following Function is used to Get Namespaces

private static XmlNamespaceManager _nsmgr;

private static XmlNamespaceManager NameSpaceMgr

{

get

{

if (_nsmgr == null)

{

_nsmgr = new XmlNamespaceManager(new NameTable());

_nsmgr.AddNamespace(“wss”, “http://schemas.microsoft.com/sharepoint/soap/”);

_nsmgr.AddNamespace(“s”, “uuid:BDC6E3F0-6DA3-11d1-A2A3-00AA00C14882”);

_nsmgr.AddNamespace(“dt”, “uuid:C2F41010-65B3-11d1-A29F-00AA00C14882”);

_nsmgr.AddNamespace(“rs”, “urn:schemas-microsoft-com:rowset”);

_nsmgr.AddNamespace(“z”, “#RowsetSchema”);

}

return _nsmgr;

}

}

private Type TypeFromField(XmlElement field)

{

switch (field.GetAttribute(“Type”))

{

case “DateTime”:

return typeof(DateTime);

case “Integer”:

return typeof(int);

case “Number”:

return typeof(float);

default:

return typeof(string);

}

}

Tech in the 603, The Granite State Hacker

The Great Commandment

While I was writing a post the other day, I noticed that I had neglected a topic that I find very important in software development. Risk management.

There are only a few guarantees in life. One of them is risk. Companies profit by seizing the opportunities that risks afford. Of course, they suffer loss by incidents of unmitigated risks. All our government and social systems are devices of risk management. In business, risk management is (now, and ever shall be) the great commandment.

Many software engineers forget that risk management is not just for PM’s. In fact, software and its development is fundamentally a tool of business, and, by extension, risk management. The practice of risk management in software really extends in to every expression in every line of source code.

Don’t believe me? Think of it this way… If it wasn’t a risk, it would be implemented as hardware. I’ve often heard hardware engineers say that anything that can be done in software can be done in hardware, and it will run faster. Usually, if a solution is some of the following…
· mature,
· ubiquitous,
· standard,
· well-known,
· fundamentally integral to its working environment

…it is probably low risk, particularly for change. It can likely be cost-effectively cast in stone (or silicone). (And there are plenty of examples of that… It’s what ASIC’s are all about.)

Software, on the other hand, is not usually so much of any of those things. Typically, it involves solutions which are…
· proprietary,
· highly customized,
· integration points,
· inconsistently deployed,
· relatively complex / error-prone
· immature or still evolving

These are all risk indicators for change. I don’t care what IT guys say… software is much easier to change than logic gates on silicone.

I’ve dug in to this in the past, and will dig in more on this in future posts, but when I refer to the “great commandment”, this is what I mean.