Most DTP vendor evaluations rely on methods that predict the wrong thing: reference checks that describe past performance for other clients, portfolio reviews that show finished work with no visibility into the process behind it, and RFP responses that describe capability in the vendor’s own words. None of these test what actually matters, whether the vendor’s process holds up on your specific files, your specific CAT tool, and your specific languages. The one-project test does that directly: send one real, representative file, and let the result speak for itself.
This matters for Vendor Managers choosing between evaluation methods before committing to a new DTP vendor or adding one to an Approved Vendor List.
Why the common evaluation methods fall short
Reference checks tell you how a vendor performed for someone else, on files that may look nothing like yours. A reference who works exclusively with simple text documents cannot speak to how a vendor handles complex InDesign files with nested styles, because that reference never tested it.
Portfolio reviews show finished output, polished and presented at its best. They reveal almost nothing about the process that produced it: how many revision cycles it took, whether the CAT tool round-trip stayed clean, or how the vendor handled the parts that did not go smoothly. A portfolio is a highlight reel, not a process audit.
RFP responses describe capability in the vendor’s own language, unverified. “Compatible with major CAT tools” and “experienced with complex layouts” are claims, not evidence. A well-written RFP response correlates with writing skill more than with actual production quality.
Each of these methods answers a real question. None of them answers the one that predicts what will actually happen on your first project: does this vendor’s process hold up on a file like the ones you actually send.
There is a reason all three remain standard practice despite this gap. They are fast, low-cost, and do not require committing a real project to find out. That trade-off is real, and it is exactly why the one-project test should not replace them entirely. It should come after them, as the step that turns a shortlist into a decision.
What the one-project test actually is
Send a real file, not a simplified sample built to be easy. Do not tell the vendor what you are specifically checking for. Then evaluate what comes back against the same standard you would hold any delivered project to.
The test works because it removes the gap between what a vendor claims and what a vendor’s process actually produces under normal working conditions. A vendor cannot talk their way around a file that came back with broken tags or a layout that fell apart under text expansion. The file either holds up or it does not.
This only works if the test happens under conditions close to a real project. A vendor who knows exactly what is being tested and treats the file with unusual care is not showing you their normal process, they are showing you their best possible effort on a known evaluation. That is why not announcing the specific evaluation criteria matters more than it might seem. The point is to see what a normal project looks like, not what a vendor produces when they know they are being graded on it.
How to choose the file you send
The file needs real complexity, matched to what you actually need done. If your typical work involves multi-language layouts, send a file with more than one target language, not a single-language sample. If your work regularly includes tables, forms, or complex formatting, include those elements. A file that is easier than your typical project tests nothing you actually need answered.
Do not send your hardest possible file either. The goal is a representative test, not a worst-case stress test that could fail even a strong vendor on an edge case unlikely to reflect ongoing work. Representative, not extreme, in either direction.
What to look for when the file comes back
Does the file need a QA cycle before it can go to a client, or is it publication-ready on the first handback. Does your own team have to fix anything before it is usable. Does the vendor demonstrate they understand expansion percentages for your target languages without being told, or do you have to explain layout explosions from scratch. Does the file survive re-import into your CAT tool without corrupting a single tag, with fuzzy match rates against your existing translation memory holding where they should.
Beyond the file itself, watch how the vendor communicates about it. Do they proactively flag anything unusual about the source file, or do they simply return output without comment. A vendor who notices and raises a structural issue in your source document is showing you how they will handle problems on a live project, which is exactly the information a portfolio review cannot give you.
Two vendors can return technically similar files with very different signals underneath. One returns the file silently, no comments, no flags. The other returns the file with a short note: the source document had inconsistent heading styles that were normalized during layout, and a table on page four had merged cells that needed to be reconstructed manually. The second vendor is showing you an actual quality process. The first may have done the same work and simply not mentioned it, or may have missed it entirely. You cannot tell the difference from the file alone, which is exactly why the communication around the test matters as much as the file itself.
Comparing the methods directly
Reference checks and portfolio reviews are useful for a first filter, narrowing a long list down to a few vendors worth testing further. They are weak as a final decision method, because they describe past performance for someone else’s files, not predicted performance on yours. Keep them in the process, but keep them in the right place in the sequence.
RFP responses are useful for confirming a vendor covers the basic requirements, format range, language pairs, general capability, before investing time in a real test. They should never be the sole basis for a final decision, because a well-written response and a well-executed project are not the same thing.
The one-project test is the only method in this list that tests actual production output under real conditions. It costs more time upfront than reading a portfolio, and it is the only one of the four that answers the specific question a Vendor Manager actually needs answered before committing ongoing volume to a new vendor. That trade, a small amount of extra time before commitment against a much lower risk of an unpleasant surprise on the first real project, is usually a clear one once it is stated plainly.
Running this as part of a structured evaluation
A practical sequence: use RFP responses and references to build a shortlist of two or three vendors worth a real test. Send each of them the same representative file, without telling them what you are checking for. Compare the results against each other, not just against an abstract standard, since a side-by-side comparison surfaces differences a single test in isolation can miss, particularly on judgment calls where there is no single objectively correct answer, only a better and a worse one.
This sequence is not slower than a traditional RFP process in practice, because it replaces weeks of reference calls and portfolio review with a test that produces a direct answer in the time it takes to complete one project.
There is a cost consideration worth addressing directly. A real test project means paying for real work, not a free sample. Some vendors resist testing on those terms, treating a paid evaluation project as a sales expense rather than a legitimate part of the sales process. That resistance is itself useful information. A vendor confident in their process treats a paid test as an opportunity to demonstrate it, not a burden to negotiate around.
What this looks like when Foliage is the vendor being tested
We welcome this test specifically because it is the fastest way to demonstrate what a written capability claim cannot. Send a representative file, and the file that comes back will show format handling, CAT-tool round-trip integrity, and adherence to a Clean Delivery standard directly, without asking you to take any of it on faith.
We also treat the test itself as evidence of process, not just a hoop to clear before winning business. The same QA steps, the same file preparation checks, the same communication about anything unusual in the source document, apply to a test project exactly as they would to ongoing volume. What you see in the test is what ongoing work actually looks like, not a version dressed up specifically for evaluation.
Talk to Foliage Solutions about running a one-project test on your next multilingual DTP evaluation. One file tells you more than a stack of reference calls ever will.