Migrations – when “common sense” isn’t
Summary: Use a tool, over developing scripts, for moving documents and content to SharePoint Online. Overall it's quicker, cheaper, with lower-risk and higher-chance of success.
Its common-sense isn't it...
We have been involved in a lot of moves, migrations and upgrades over the years and it astounds me that people short-change investing in appropriate tooling to support it.
The cost-avoidance is usually because some of these toolsets and suites have very large price-tags attached, and immediately fail the ‘gulp-test’ …otherwise known as “sticker-shock” (look it up kids).
Unfortunately, we often see document migration tools presented for only 1 thing, and that is short-sighted because there are a number of gaps in release management (UAT to Production, and back) and migration support that the same tools will support.
Cloud has commoditised a lot of services, software and tools in last few years, not least of which are migration tools. Where things remain cloudy (pun intended) is the evolving means by which we pay or price toolsets.
More and more we see a combination of pricing options – most of which don’t align traditional project capital expenditure model and so are missed:
i. Single-shot cost for basic functionality
ii. Organisation / tenant subscription cost – annual
iii. Organisation / tenant subscription cost – quarterly
iv. Consultant / professional service org cost + pass-through margin
v. Online volume-data move - $/GB service
Why worry? Don’t we just need to copy-n-paste?
If you are moving content from file-shares then to an extent you can, but it is not really appropriate with business owned content.
You would not typically worry about metadata, version history, workflow etc for employee's personal drives e.g. "Home drive" docs.
A basic approach which "copy-and-pastes" documents makes sense - plus there free scripts (e.g. from Microsoft) or the numerous examples of powershell available.
Unfortunately this does not work when we need to protect the information and identify/keep/maintain lifecycle info about the documents which are business critical.
Where’s the value in paying for a tool? Just write-it yourself…?
We repeatedly see professional services and IT shops charging a lot of money to “custom build” migration tools and scripts for customers document migrations.
While I know of at least 1 scenario this is valid for, I fervently believe it is mostly unnecessary today, and sometimes just blatant gouging of the customer.
To quote the adage: "it is tempting, if the only tool you have is a hammer, to treat everything as if it were a nail."
Abraham Maslow, 1966
If you have developer, then most of the options you get are “build-it-yourself”. This works out great for the consultants, not so good for customers who have to pay for the constant re-writing of tools.
If you offer me a lot of money to script-up tooling for you, I would be happy to take the money – as long as you understand that this is will go on a holiday and drinks and I will then use one of the off-the-shelf tools which has:
i. Online training
ii. Support and Knowledge base articles
iii. Community exchange
iv. Patches, fixes and access to a product development team
v. Built-in analysis tools, reporting and support for multiple version releases as both source/target
vi. Works.
…and costs the equivalent of approx. 4-man days ($3,995USD or $5690 NZD)
This estimation of man-time equivalent is based on:
i. the most-expensive but comprehensive tool in market today, with a 1-year annual subscription for 1 person;
ii. using NZ industry average IT consulting charge-out rate of $175/hr and
iii. Bing currency converter for USD-to-NZD rates on 12/06/18
BUT I have a developer handy?!
Past experience, and freely available online resources, means I feel very comfortable in saying a good SharePoint / CSOM developer with PowerShell experience can knock-up a file copy and upload script (basic) copying from 1 specific source to 1 specific target - and have it tested in about 4 – 6 hours.
I can also justify saying that you will invest a further 16+ hrs in ensuring something more than just a copy-n-paste script to build out testing in new environment, (re)creating libraries, folders etc. So, I’m at approx. 20hrs.
Calculation: 20 * $175 = $3,500
On the assumption that there are only minor bugs and issues, you may get up to 28 hrs ($4,900) and be ready to go... before actually migrating anything.
I do not considered this cost-effective, if we identify the ongoing ownership of DIY:
No bug fixes - except under in T&M by customer
I own debugging effort, when it has issues
No “what if” reporting, or analysis on source.
A re-write for new features e.g. move between different sources, or copy in same-source
No external community or support engineers to check with
No volume testing – until real exercise
Validation will be command line error-code and visual review
No training for end-user/sme in customer
No support calls or ticket mgmt. for customer
No documentation other than the developer notes (not guaranteed)…
I’m only moving some documents…
No, you aren’t. Let me show you why. The following are real migration requirements outside of the copy-n-paste of documents:
1. From the source environment I want to retain my: created (date), created by (name), modified (date), modified by (name) – in the target
…err – more development
2. From the source environment I had information on folder names e.g. Financial year folders under Accounts. In new target I want that imbedded as metadata.
…err – more development (a lot)
3. Can I have a validation report of: what went, what failed and why?
…err – more development and building in event-handlers and error trapping
4. From the source, I want to take my version history on my documents to the new environment*
…err – more development, and a big headache
*this one is specific to migrating from a DMS, or CMS not file-share
5. In the target environment can I move groups of documents and folders around to restructure my docs – but not lose name-ownership/modified by metadata
…err – more development
[in SPO] the OOB move or copy commands don't preserve metadata
…and those are just 5 of the most common requirements I have heard in every business document mgmt migration from the last 5 years.
By the time you factor a number of these enhancements in to your “custom scripts” you will be heading well north of 40 hours (min). ..which is ~$7000 NZD
Cloud continues to change things…
Based on the above, even just to do a basic bulk-move of documents the options comparison is too close to call:
PRODUCT $5689 vs. SCRIPT $4900
Remember this is a practical, experience based estimate on getting the tools in place. Once we add in effort to run the migration activity manual script effort will go up (~$7000). Does scripting-it still seem good?
Now compare this to what I saw quoted recently by one experienced and "certified" migration provider:
"Build of custom scripts for copying documents to SharePoint Online - $10k – $50k, time and materials."
The range suggests the degree of uncertainty and risk being assumed by the provider. The estimate is only for creating the tools. NOT moving the content.
But changes and availability in market over the last 5 years have driven:
Consistency in access methods, tools and standard practices
A multitude of tools and services available in a commodity area
Evolving pricing and access models – one-time purchase, volume moved, monthly/quarterly/annual subscription
Continual downward spiral of costs
From a project perspective, $5689 is trivial cost to offset risks of things going wrong with content move *and* to get the surety of upfront analysis, reporting and validation.
It still costs more for the tool...
Remember I did the calculation using the most expensive of the cheap options for the calculations.
I recently did an assessment for one customer – looking at the top 3 (out of 20+ options) plus this “best in class” provider:
[remember: best of the cheap options – not enterprise software suites]
A stock tool which only copies documents and creates file-structures – #700 GBP, or $1334 NZD (single purchase)
A professional tool which allows you to run a analysis before copy –$1200USD, or $1709 NZD annual subscription
A professional tool which understands mixed source and target repositories, maps attributes - $2500USD, or $3418 NZD (per quarter year)
The best-in-class tool which does all of the above, plus allows for data remapping, test run-through and various options for expanding - $3995 USD, or 5689 NZD (annual subscription)
..don’t believe me?
Pricing is not the issue. Its the choice of if we want manage the risks vs. unnecessary labour costs for what is a common task.
If tools are so cheap, why this article?
Over the life of migration project, even a simple one from file-shares to SharePoint online, the effort and cost is not in a tool.
The real effort and cost is really understanding your content, business processes and get in to the right frame-of-mind for cleaning-up the old junk that’s not needed.
The cost of tools and support has ranged from 6.5% to 13.2% of the overall cost of the project. Heck, the cost of project management us usually 2 – 3 times that.
statistics range from analysis of 20+ migration projects over 5 years
I suspect what we are seeing with professional services firms is
the “I have a hammer…” syndrome, or a degree of revenue protectionism “I will keep my people busy… and its t&m”, what ever the case it is negligence to ignore the market and changes around it.
Perhaps I'm being a little harsh, but even if I am it's still not good practices that we are seeing.
Why write this?
We work with everyone from 15-people agencies, to multinational companies with x000s of staff. The big companies have IT in-house – they employ teams with highly intelligent people and time to do things, but smaller organisations need support and often don't have a lot of money.
From a consulting perspective we are here to make money and do interesting jobs that offer value to our customers. That doesn’t mean being lazy. It doesn't mean gouging.
Yes, consulting companies might lose some money on switching in tools for code, but they get consistency, reduced risk, predictable margins *and* can help the customer focus on value.
Best of all there’s less to go wrong! …because we all know projects have problems and minimising the risk is what we should be doing. Predictability on margins and reduced risk spell improved profit to me, and greater chance of success in delivery (and reputation).
So it really can pay to keep up to date, and not just in development experience.
Want to talk about Migration planning, and best practices? Drop us a line: hi@timewespoke.com