In Short: What Should You Test For?
Modelling judgement, not DAX trivia. Most candidates can build a report; far fewer can design a model that still works in three years. Give them a badly-designed model and ask what they would change and why - it separates report builders from modellers in about ten minutes. And before writing the job description, be honest about whether the work would actually fill someone's week.
Three Skills, Rarely in One Person
"Power BI developer" covers at least three distinct capabilities, and job adverts routinely ask for all three at one salary.
Report building. Visual design, understanding what a user is trying to decide, restraint about how much goes on a page. Most common, most visible, easiest to assess.
Data modelling. Star schemas, relationships, grain, DAX beyond basic aggregation. Much rarer, much harder to assess, and considerably more consequential - the model determines whether your estate is fast and consistent or slow and contradictory.
Platform. Refresh and gateways, workspaces and deployment, security, capacity. Often entirely absent in candidates from analyst backgrounds, and it is what stops your estate breaking at 3am.
Decide which you actually need before advertising. A team with a good model and no reports needs a report builder. A team with forty reports that disagree needs a modeller. Those are different hires at different price points, and advertising for one while expecting the other is the most common cause of a failed hire in this field.
Interview Tasks That Work
Give them a bad model. Prepare a small model with recognisable problems - a flat wide table, a missing date dimension, bidirectional relationships everywhere, calculated columns doing a measure's job. Ask what they would change and why.
This is the single most revealing exercise available. A report builder will comment on the visuals. A modeller will restructure it and explain the consequences. You will know within ten minutes which you are talking to.
Ask about a report they made faster. Then ask what the actual cause was. Strong answers name a specific cause - cardinality, a measure scanning too much, too many visuals - and describe how they diagnosed it. Weak answers describe changes they made hoping something would help. This tests diagnostic thinking, which is the skill that matters when something breaks. See report performance for what a good answer sounds like.
Pose a definitional conflict. "Sales and finance define revenue differently. What do you do?" You are testing whether they see this as a business problem to facilitate or a technical problem to work around. The technical answer - build both - is the wrong one, and it is the common one.
Ask what they would not build. Good candidates have opinions about scope and can describe a time they pushed back. Candidates who would build anything asked of them produce estates nobody can maintain.
What Not to Test
DAX syntax trivia. Anyone can look up a function. Knowing when to use a variable, or why a measure is scanning more than it needs to, is judgement - and that is what you should probe.
Certification alone. Certifications demonstrate study, not delivery. Useful as a filter at volume, weak as a signal on its own.
Visual polish in isolation. A beautiful report on a broken model is a liability, because it is trusted more than it deserves to be.
Do You Need the Hire?
Worth answering honestly before recruiting, because the most expensive hiring mistake is hiring for work that does not exist.
Hire when the work would genuinely fill the week. If reporting demand is continuous and substantial, a permanent person is more responsive and cheaper than any alternative.
Hire when business context dominates. In organisations with unusual domain logic or heavy regulation, someone living in the business will outperform an external specialist, whatever their platform depth.
Do not hire yet when demand is two days a week with month-end spikes. That is a common shape and a poor fit for a full-time role. A retained service or a fractional arrangement covers it better - see managed service versus hiring.
Do not hire yet when you do not know what the role should be. A year of a retained service teaches you exactly what your permanent hire should look like, which is far clearer than guessing beforehand.
Setting Them Up to Succeed
If you do hire, three things make the difference:
Define success in business terms before they start. Reports people actually use, questions answered without escalation, refreshes that stop failing. Measuring output in dashboards produced rewards volume over quality and produces an unmaintainable estate.
Give them ownership of the model, not just the reports. If the model is owned elsewhere or by nobody, a good modeller will leave.
Budget for external depth occasionally. One person cannot be expert in modelling, platform administration, governance and the source systems. A hybrid - internal ownership with external specialist support - is what most successful mid-market teams settle on.
Where Solv Systems Comes In
We are not a recruiter and we do not place people. We are frequently asked to help assess candidates, and we are happy to - an hour of technical screening from someone who builds these estates is usually worth more than another interview round.
We also cover the gap while organisations recruit, and we say plainly when hiring is the better answer than retaining us. If the work would fill someone's week and your business context is the hard part, hire - and we would rather tell you that than sell a retainer that quietly becomes a substitute for the decision.
Sources and Further Reading
Frequently asked
Three layers, and candidates are rarely strong in all. Report building - visual design and user empathy. Data modelling - star schemas, relationships, DAX beyond basic aggregation. Platform - refresh, gateways, workspaces, security. Most candidates present the first well; the second is what separates them, and the third is often missing entirely.
A report builder makes good-looking, useful reports on a model somebody else designed. A modeller designs the model everything else depends on. The second is rarer, harder to assess in interview, and has considerably more impact on whether your estate still works in three years.
Give them a badly-designed model and ask what they would change and why - it reveals modelling judgement in minutes. Ask about a report they made faster and what the actual cause was. Ask how they would handle a metric two departments define differently. Avoid DAX trivia; anyone can look up syntax.
Only if the work would genuinely fill their week. Many organisations need perhaps two days a week, spiking at month-end - which is a poor fit for a full-time role and a good fit for a retained service. Be honest about the volume before writing the job description.
Rates vary widely by market and by which of the three skill layers you actually need. A report builder and a senior modeller are different roles at different price points, and advertising for one while expecting the other is the most common cause of a failed hire in this field.
Set the measure before they start, in business terms: reports people use, questions answered without escalation, refreshes that stop failing. Output measured in dashboards produced rewards the wrong behaviour and produces an estate nobody can maintain.
