DavidOverton.com
This site is my way to share my views and general business and IT information with you about Microsoft, IT solutions for ISVs, technologists and businesses, large and small.  
Who should test software and service packs - I think vendors,customers and partners - others think it is just Microsoft (vendors)

I saw this in a forum and got a bit upset by it... it read:

“SP2  is not going anywhere near anything we manage until some other people have tried it and seen whether it has any unexpected side effects.”

 This was followed by

“What would happen if everyone took that attitude? I believe that we have to take responsibility for testing these service packs ourselves for the benefit of our customers, not waiting for someone else to try it first.”

And then

“That's MS's job - not ours.“

And

“It's not about whether others can be bothered it's about whose job it is to test these patches. It's not mine, nor my customers.

There are far too many Microsoft bashers and there are far too many Microsoft sycophants - the rest of us just get shouted down when we say it like we see it.”

The last comment really got my goat, so much so that I posted back to the forum this reply, but I thought I would make it wider as I think the conversation should be discussed broader.  Now in defence, I have snipped little bits of the conversations to highlight the point and people posted things like:

“And do you think MS didn't test it? Give them a break.
I am of the opinion that as soon as something new comes out you go like this:
First week
On your test box (or if none then home system)
Second week
On your own production domain if there has been no 'blog storm'
Third week
On 10-30% of your clients
Fourth week onwards
Slowly install on the remains of your client base until after 10 weeks or so all is done

Key thing is to keep an eye on the blogs and SBS2K Yahoo group”

And

“Well I am with you Ian on this - its exactly what I do - test, test, test, check, verify, and test!
IF other cannot be bothered, leave them to it !”

 

In the end, this is what I posted.  Flame me if you disagree, comment either way, but I think that IT professionals are part of solution and that means testing software, not on behalf of the vendor, but on behalf of the customer who should not have to pay IF something is rolled out and is untested and does not work as expected.

My Post to the UKSBSG group:

I don't know which group I fall into, but I do disagree with your statement that it is neither you nor your customers role to test a Service Pack.

I'm sure I'm going to put some noses out of joint here, so I should say this is David's view, not necessarily Microsoft’s.  I also have a banging headache that won't go away and I have "coffee shop" music playing in my ear which has included “Chariots of fire” and “Hawaii five-o”, so I could blame my "strong" views on that.

Microsoft builds service packs to fix a number of problems and sometimes to change the behaviour of a server.  This is tested extensively internally and externally and then released with no known issues.  They are not always perfect and we need to be told about the things we do not find.  Thousands of vendors (non-Microsoft people) are involved in the signing off of the product to try to get the best quality product out the door.

We do not test against the millions of combinations of hardware, not the 100s of millions of possible combinations of software / custom devs on a box - this is not possible.  (I guess we could charge to do that as other organisations do).  In every large corporation I have worked the IT department saw its job as taking the software as supplied by any vendor and implementing it inside the organisation to benefit the business.  Part of this process is finding all the little things that don't quite work right and making them work right by either working around them, filling the gaps or working with the vendors to fix the problems (lets call this testing).  This is what IT departments do (besides then keeping it running as it shows it still has niggles after deployment).  They don't open a box, install the software and walk away - they add value.  Part of that is knowing which box to open, but far more is doing the magic of making IT fit the business needs by whatever means necessary and then keeping it running.

 

Just because I have worked with organisations of 250,000 employees or more does not mean that these principles do not scale right down to a 1 man band business.  Someone has to work out which box to open and then make the software in the box do what the business needs, integrating it with what is already there.

I believed this is what you do.

Now this does not mean that you implement SP2 when it 1st ships, you work out whether you should look at it.  You work out the case if you don't look at it.  You could even say, I'll let someone else test it 1st, but you should still work out how key it is for you before you make that decision.

My job is NOT to test the service packs.  I don't have a technical role at Microsoft, I sit in sales - yes sales.  My job is to help a number of sales channels sell more.  I do however run a SBS box that runs several small ventures and help partners.  I installed the service pack because some of its benefits were for me.  I documented it (and the issues I found) to give others a true view of what it did for me (BTW, SQL does run faster on my box with it).

My advice, unless you always want to be 2 steps behind your fellow consultants, is to test it.  I like the approach mentioned above in the thread.  To decide which customer gets the product 1st you think who will benefit 1st or be happy to help kick the tyres and start with them.

So, opinion shared.  I think it IS your job to do some testing.  Customers probably think it is your job too.  You may want to ask them to see what they think about service packs and ultimately who they think should manage it if it does not work.  Even if you wait a bit, you still have to test it!

I'm sure others will disagree, but unless we tested solutions before we implemented them, what value are we adding as a profession?  Would you be happy if a doctor gave you a drug because lots of other people had used it and he had not heard of any problems, or if he said "you are the 1st and I think it will be good for you, but I can't be sure" and then for others "I have used this lots and it has helped many people".  I like people who test.  They tend to also be the people who do backups and know how to restore them.

 

ttfn

David

Published with BlogMailr


Posted Thu, Mar 22 2007 5:32 PM by David Overton

Comments

JB wrote re: Who should test software and service packs - I think vendors,customers and partners - others think it is just Microsoft (vendors)
on Mon, Sep 24 2007 3:10 PM

You determine every application and situation that 4 Billion people could possibly ever do or attempt to do, then test them all.

David Overton wrote re: Who should test software and service packs - I think vendors,customers and partners - others think it is just Microsoft (vendors)
on Mon, Sep 24 2007 3:35 PM

JB,

I suspect your showing my favourite form of humour - sarcasm, but if not then I believe Microsoft's answer would be:

unfortunately, we don't support the apps, the creators do, so even if we say it works, if the vendor does not, we can't override this :-)

ttfn

David

Add a Comment

(required)
(optional)
(required)
Remember Me?

(c)David Overton 2006-23