CRO Agency vs In-House Team: Which Is Right for Your Shopify Store?

Most Shopify brands doing £1m to £10m in revenue eventually hit the same wall. Traffic is healthy. Paid spend is going up. But conversion rate is flat, and the team is stretched too thin to do anything meaningful about it.

The obvious next move is to invest in CRO. The harder question is where that investment actually goes.

Hire an agency, and you get speed, specialist skills, and a testing programme that starts quickly. Build in-house, and you get deeper product knowledge, tighter brand control, and capability that compounds over time. Both arguments have genuine merit. The problem is most articles on this topic present them as equally valid for every business, which is not true.

The real question is not which model is better. It is which model is right for your store, at your revenue level, with your current team.

This article lays out a practical decision framework. It covers what each model actually costs, the traffic and revenue thresholds that change the maths, and the situations where one clearly wins over the other.

What CRO Actually Requires

Before comparing models, it helps to be clear about what a proper CRO programme involves. This matters because the common misconception, that CRO is just A/B testing button colours, leads brands to underestimate what they are actually buying or building.

A functioning CRO programme needs five things working together:

• Research and analysis: Session recordings, heatmaps, funnel data, user surveys, and customer interview insights to identify where and why visitors are not converting
• Hypothesis development: Turning data into structured test ideas with a clear rationale and expected impact
• Design and UX: Creating test variants that are properly considered, not just cosmetically different
• Development: Building and QA-ing those variants on the live store without breaking anything
• Statistical analysis: Reading results correctly, understanding sample size requirements, and knowing when a result is actually significant

That is a broad skill set. A single hire rarely covers all five. A good CRO manager might handle research and hypothesis development but need a designer and developer to execute. A developer can build tests but has no framework for deciding what to test. This is the core tension in the in-house model, and it is worth understanding before you start comparing costs.

The minimum viable in-house CRO team, according to industry benchmarks, is one experiment lead, a part-time analyst, a designer, and a front-end developer. That is four people, even if some are part-time.

For most Shopify brands under £5m, that team does not exist yet. The question then becomes whether it makes more sense to build it, or to access it through an agency.

The Real Cost of Each Model

Cost comparisons in this space are often misleading because they compare agency retainer fees against a single salary. That is not an honest comparison. Here is what each model actually costs.

Building in-house

The Adlib ecommerce salary guide for 2025 puts UK CRO salaries at:

A realistic in-house setup for a £3m-£10m Shopify brand would typically involve a CRO Manager (£52k mid-range) plus shared time from a designer and developer who have other responsibilities. Add employer NI contributions (roughly 13.8%), pension, benefits, and recruitment costs, and the true annual cost of that CRO Manager alone is closer to £65,000 to £70,000.

Then add tooling. A proper testing and analytics stack, covering an A/B testing platform, heatmapping, session recording, and survey tools, typically runs £8,000 to £20,000 per year. Enterprise platforms like Optimizely or VWO sit at the higher end.

The realistic all-in cost of a lean in-house CRO function: £80,000 to £120,000 per year, before you account for the designer and developer time being diverted from other work.

Hiring a CRO agency

UK agency retainers vary considerably by scope. Based on current market rates:

For a Shopify brand doing £1m to £5m, a structured agency programme typically sits in the £3,000 to £8,000 per month range, or £36,000 to £96,000 per year. That includes the equivalent of the full team: strategist, designer, developer, and analyst.

The comparison is not as simple as agency fee vs. salary. At the £1m to £3m revenue level, an agency almost always delivers more testing capacity per pound spent than an equivalent in-house hire. The crossover point, where in-house starts to make financial sense, is closer to £8m to £10m in revenue, when testing volume is high enough to keep a dedicated team fully occupied.

The Traffic Threshold That Changes Everything

Cost is one part of the equation. Traffic is the other, and it is often the deciding factor that gets ignored.

A/B testing only produces statistically reliable results above a certain volume. According to Littledata's Shopify benchmarks, conversion rates become statistically meaningful with at least 500 sessions per measurement period, but that is the floor for basic tracking. For a proper A/B test at a 2% baseline conversion rate, you need roughly 50,000 visitors per variant, which means 100,000 total sessions for a single two-variant test.

This has a direct bearing on which model makes sense.

Below 50,000 monthly sessions

At this traffic level, traditional A/B testing is slow and often inconclusive. Tests take weeks or months to reach significance. Running multiple concurrent tests is not viable.

This does not mean CRO is pointless. It means the work shifts from quantitative testing to qualitative research: heatmaps, session recordings, user surveys, and expert UX reviews. An agency experienced in lower-traffic CRO will recognise this and adjust the programme accordingly.

An in-house hire at this stage is almost certainly a mistake. You are paying a full salary for someone who cannot run the volume of tests needed to justify the investment. The agency model, at a lower monthly cost, gives you access to the same research skills and a more realistic programme for your traffic level.

50,000 to 200,000 monthly sessions

This is the sweet spot for agency CRO. You have enough traffic to run meaningful tests, but not so much volume that you need a large internal team to keep pace. A well-structured agency programme can run 4 to 8 tests per month at this level, with results that reach significance within 2 to 4 weeks.

For most Shopify brands doing £2m to £8m in revenue, this is where they sit. The average Shopify store converts at 1.4%, while top 20% performers hit 3.2% or above.

The gap between those two numbers, at £3m revenue, is worth roughly £54,000 in additional annual revenue for every 0.1 percentage point of conversion rate improvement. That context makes a £4,000 to £6,000 monthly agency retainer look very different.

Above 200,000 monthly sessions

At high traffic volumes, the maths starts to shift. You have enough sessions to run tests quickly, across multiple pages simultaneously, with results in days rather than weeks. A dedicated in-house team can keep pace with that velocity in a way an agency retainer cannot always match.

This is also where institutional knowledge becomes genuinely valuable. An in-house team builds a deep understanding of your customer base, your seasonal patterns, and your product catalogue over time. That context produces better hypotheses. At this scale, the long-term compounding value of in-house expertise starts to outweigh the flexibility advantage of an agency.

When a CRO Agency Is the Better Bet

An agency makes sense in more situations than most brands expect. The case is strongest when one or more of the following applies.

You do not have execution capacity in-house

This is the most common scenario. The marketing manager wants to run tests. The developer is committed to other projects. The designer is already stretched. The result is a CRO backlog that never moves.

An agency brings its own execution team. Tests get built, QA'd, and launched without competing for internal resource. For brands where conversion work keeps getting deprioritised, this alone justifies the retainer.

You need results within six months

Building an in-house CRO capability takes time. Hiring takes two to four months. Onboarding takes another one to two months. Getting a testing programme to full velocity takes longer still.

An agency can typically begin research in week one and launch the first test within four to six weeks. If you are heading into peak season, launching a new product range, or have a specific revenue target to hit, the agency timeline is not just convenient, it is often the only viable option.

Your store is below £5m in revenue

At this revenue level, the maths almost always favours an agency. The cost of a properly resourced in-house function (salary, NI, benefits, tooling) exceeds what a good agency programme costs, while delivering less testing capacity and narrower expertise. The exception is if you already have a designer and developer in-house who can be partially allocated to CRO work, which changes the cost calculation significantly.

You want cross-industry pattern recognition

An agency working across 10 to 20 Shopify clients at any one time builds a pattern library that no single in-house team can replicate. They have seen what works in your category, what has failed on stores with your traffic profile, and what tests are worth running versus which ones rarely move the needle. That institutional knowledge is part of what you are paying for.

When In-House Wins

The in-house model has a genuine case, but it is a narrower one than many brands assume. It becomes the right answer when these conditions are in place.

You are running more than eight tests per month

At this testing velocity, you have outgrown what most agency retainers are structured to deliver. The overhead of briefing an external team, reviewing work, and managing the relationship adds friction that slows things down. An embedded team, working in your Slack channels, attending your trading meetings, and living inside your Shopify admin, can move faster.

You have a complex product catalogue or brand

Some stores are genuinely hard for an external team to get up to speed on. Highly technical products, regulated categories, or brands with very specific voice and visual standards take longer to onboard. If getting an agency to the point of producing good work takes six months of hand-holding, the productivity argument weakens.

You are above £8m to £10m in revenue and scaling

At this revenue level, the total cost of ownership of an in-house team becomes competitive with a senior agency retainer. More importantly, the volume of work, across product pages, collections, landing pages, checkout flows, and email journeys, is enough to keep a dedicated team fully occupied. The compounding value of institutional knowledge also starts to tell. An in-house team that has been running your CRO programme for two years will have a hypothesis backlog, a test history, and a customer understanding that no agency can replicate.

You already have the design and development resource

If your existing team includes a designer and developer who have capacity, the incremental cost of adding a CRO Manager or Strategist is significantly lower. You are not building a team from scratch; you are adding the strategic layer on top of existing execution capability. In this scenario, a mid-level CRO hire at £52,000 to £65,000 can unlock the capacity you already have.

The honest objection to in-house: most brands that say they want to build in-house capability underestimate how long it takes and how much management attention it requires. Hiring, onboarding, and retaining a good CRO Manager is a meaningful commitment. If the head of ecommerce is already at capacity, adding a specialist hire who needs direction and mentorship can create more problems than it solves.

The Decision Framework: A Practical Guide

Rather than a generic pros and cons list, here is a set of specific criteria to run your situation against.

The hybrid model worth considering

A third option that often gets overlooked is starting with an agency and building in-house over time. This is not a compromise; it is often the most strategically sound approach.

An agency can run your CRO programme for 12 to 18 months, establish a testing methodology, build a hypothesis backlog, and prove the revenue impact. That track record makes the business case for an in-house hire far easier to make internally. It also means the person you eventually hire steps into a functioning programme rather than starting from scratch.

Many brands find that the agency relationship does not end when the in-house hire arrives. Instead, the agency shifts to a lighter advisory or sprint-based model, providing external perspective and specialist capacity for larger projects while the in-house team handles the day-to-day testing programme.

What to Do Next

The agency vs. in-house question is ultimately a resource allocation decision, not a philosophical one. The right answer depends on your traffic volume, your revenue, your existing team, and how quickly you need results.

For most Shopify brands doing £1m to £8m in revenue, the numbers point clearly towards an agency, at least as the starting point. The cost is lower, the speed to results is faster, and the breadth of expertise is wider than a single in-house hire can provide. The in-house model earns its place when testing velocity is high, when the team already has design and development capacity, and when the business is at a scale where long-term institutional knowledge compounds into a genuine competitive advantage.

The worst outcome is doing neither. A flat conversion rate while ad spend increases is a revenue leak that compounds quietly. Every month without a structured CRO programme is traffic that could have converted better.

If you are not sure which model fits your store, a useful starting point is an honest audit of your current conversion rate against Shopify's benchmarks, your monthly session volume, and the execution capacity of your existing team. Those three numbers will tell you more than any general framework.

Futur Media works with Shopify brands on conversion rate optimisation as part of a broader performance partnership. If you want to understand where your store is losing revenue and what a structured programme would look like for your specific setup, get in touch and we can walk through it together.

‍