A small biotech is 8 weeks away from a Phase II read-out. The SAP is signed. Data management is in the process of resolving the final open queries. No one has started working on the table shells because the single SAS programmer the company hired left in April, and the job advertisement remains online.

Therefore, should the company hire or outsource? If you have one or two studies and do not have a dedicated programming team, then outsourced TLF programming is usually the better fit. If you have multiple programs running concurrently and a steady flow of work, then you need to create a team. Most companies fall somewhere in-between these two extremes.

Thus, the specifics of the situation are important. Here is how we would approach the in-house vs outsourced clinical programming decision.

What You Are Truly Paying For

TLFs (tables, listings, and figures) provide summaries of your clinical trial, including demographic information, adverse events, efficacy results, patient lists, and survival curves. TLFs are included in the interim analysis, the DSMB package, and the clinical study report.

One aspect that people often overlook is the rerunning of analyses. A study with 300 outputs does not get programmed once. Rather, it gets programmed, reviewed, and rerun each time there is an update to the data. By the time database lock occurs, the demographic table has been generated numerous times, and each iteration requires its own review. We discuss how to develop quality outputs in our blog post entitled TLF Programming Best Practices for Clinical Study Reports.

Keep the rerunning of analyses in mind. It is the primary reason why the staffing issue is more complex than it appears.

Why Some Companies Hire Their Own Staff

The Benefits of an In-House Programmer

There are several valid reasons to consider hiring your own staff. An in-house programmer is physically located next to your statistician and medical lead. Therefore, questions can be answered in a matter of minutes rather than days.

Additionally, a programmer who has previously worked on two studies involving the same compound will be knowledgeable of its peculiarities. For example, they will know why visit 4 has an unusual window and which safety concerns the DSMB continually raises. This knowledge is difficult to obtain elsewhere.

Furthermore, if your pipeline is continually producing new projects and the workload never ceases, a permanent salaried team can be more cost-effective over time than utilizing an external contractor.

The Challenges of Hiring

Hiring is challenging, however. The time required to find and onboard a new programmer can take months. Experienced statistical programmers with actual CDISC experience are rarely available on job boards for extended periods of time. If your database lock date is already established on the calendar, conducting a lengthy hiring search creates a degree of risk that you cannot manage.

Additionally, relying solely upon one programmer presents issues related to quality control. A single programmer cannot perform quality control on their own work. Independent quality control exists precisely because individuals who create a program tend to interpret it based upon their intent and not necessarily based upon its execution. As such, errors can persist until final reporting. Therefore, upon hiring your initial programmer, you may require additional hires almost immediately.

The workload is also variable. TLF generation experiences spikes surrounding interim analyses, database locks and clinical study reports. During other times, the workload is relatively low. During these quiet times, you continue to pay salaries.

Finally, you must create all necessary infrastructure, including software licenses, a validated environment, programming standards, a macro library and written procedures. An auditor will require documentation regarding all of these items.

Why Others Choose Outsourced TLF Programming

The Benefits of Outsourcing

Outsourcing provides benefits primarily related to timeliness. An established contractor with an existing team can commence operations once you provide them with your SAP, shell definitions and data specifications. You do not experience delays due to a hiring process.

Additionally, outsourcing provides a built-in backup programmer by default. Independent quality control is a standard component of how reputable contractors operate, and not something that you must staff separately. The contractor possesses established standards, macros and validation processes that significantly reduce startup time for companies that currently lack these resources. We provide additional details regarding why emerging biotech companies utilize specialized biometrics CROs in our blog post titled Why Small Biotech Companies Choose Specialized Biometrics CROs.

The Drawbacks to Watch For

While outsourcing offers numerous advantages, it also has disadvantages. Contractors who claim otherwise are attempting to market their services.

• Onboarding requires substantial effort. The contractor requires your protocol, SAP and data specifications prior to executing any tasks effectively. Missing information within these documents will quickly become apparent.
• Knowledge can exit the company. If the contractor’s code is poorly documented, you will be dependent upon them for every modification.
• Quality varies significantly. Many contractors assign a senior-level representative to attend the kickoff meeting and then assign your project to junior-level personnel. It is essential to ask directly which individuals will develop your programs and which will perform quality control.
• Contracts matter. Fixed-price arrangements, hourly rates and dedicated-resource models behave differently when your scope changes (and it will). Our blog post titled A Guide to Biometrics CRO Pricing Models describes what each model covers.

In-House vs Outsourced Clinical Programming: How to Decide

Do not engage in a generalized discussion regarding whether to hire or outsource. Instead, evaluate your specific circumstances.

1. Timing

If your next data cut is several months away, it is unlikely that hiring will allow you to meet deadlines. Outsourcing is then the most practical option, irrespective of your long-term preferences.

2. Number of Studies

One or two studies over the next two years rarely support establishing a programming team. A pipeline containing multiple ongoing studies may justify doing so, however.

3. Who Performs Quality Control

If your honest response is “the same person who wrote it,” address that issue first, regardless of which path you pursue.

4. Coverage if Your Lead Programmer Is Absent

Determine what occurs if your lead programmer is absent for one month. If your response is “the study stops,” you are assuming greater risk than your budget indicates.

5. Contract Terms

Finally, review the terms of your contract. Upon completion of the project, you should retain ownership of all programs, logs and specifications. This enables you to reuse them or transfer to another contractor without starting from zero.

A Hybrid Approach

Plenty of emerging biotech companies ultimately adopt a hybrid approach. They retain a statistician or lead programmer internally (someone who reviews shells, approves outputs and maintains knowledge about their products), while outsourcing production-level work to a contractor. The contractor assumes responsibility for volume, rerunning analyses and independent quality control.

You maintain control over the areas of importance while gaining additional capacity when data cuts occur. If your pipeline expands subsequently, you may opt to bring additional work back in-house without experiencing significant disruption.

What to Check Before Contracting With an External Partner

Ask these questions, and keep asking until you get clear answers:

• Who will perform the actual work? Continue inquiring until you obtain names and seniority levels.
• How will quality control be conducted? Find out whether critical outputs will be independently programmed or merely reviewed.
• Which software do they use? SAS is currently the industry standard for regulatory submissions, while R is gaining popularity, particularly for generating figures and performing exploratory analyses. Ask which they use and why.
• Are CDISC conformance checks executed? Confirm that software such as Pinnacle 21 is run prior to delivery of any materials.
• What occurs if you modify scope mid-project? It will happen.
• Who do you email when a table appears incorrect, and how quickly can you expect a response? The answer provides insight into how your project will proceed.

How Weltrix Works

Weltrix collaborates with emerging and mid-sized biotech sponsors, and that shapes how we run programming. Senior biostatisticians and statistical programmers manage each study, and it is not handed to junior-level staff after kickoff. You may communicate directly with the individuals responsible for completing the work.

We program in SAS Base and Macro, R and Python. All outputs undergo independent quality control, and critical outputs are developed by two independent programmers. Prior to delivery of any materials, we execute CDISC conformance checks using Pinnacle 21. Details regarding our services are available on our TLF Programming Services page.

If you only require assistance during peak periods, that is acceptable as well. We would prefer to design projects based upon your SAP and data cut dates rather than delivering packages that do not meet your requirements.

Frequently Asked Questions

What does outsourced TLF programming typically include?

Generally, it includes shell review, programming of tables, listings and figures from your analysis datasets, independent quality control, and rerunning analyses after each data cut. Some contractors also develop SDTM and ADaM datasets, define.xml and reviewer’s guides. Obtain written documentation outlining the scope of services provided.

Is outsourcing less expensive than hiring a programmer?

It depends on your workload. For one or two studies with intermittent workloads, outsourcing generally costs less because you only pay for completed work. Conversely, an in-house team may become more economical over time if you have heavy, continuous workloads throughout the year. Do not compare salary alone. Compare the total cost of hiring, software, validation and management.

Can we outsource TLFs while retaining datasets internally?

You may do so only if your ADaM datasets and specifications are clearly defined and stable. TLFs depend on the datasets, so modifications to the datasets will result in modifications to the outputs. Many sponsors prefer to delegate both tasks to a single team.

How quickly can a contractor begin work?

Once they receive your protocol, SAP, shells and data specifications. The sooner they receive these materials, the sooner programming commences.

Who owns the code generated by contractors?

You should. Ensure that your contract stipulates that you receive all programs, logs and specifications upon completion of the project, so you can reuse them or transfer to another contractor without starting from scratch.

SAS or R for TLF programming?

Both are used. SAS has historically been the standard for regulatory submissions, while R is gaining traction, particularly for generating figures and performing exploratory analyses. Select based on your existing code, regulators and team, and agree on the choice at the start so the study remains consistent.

When should a biotech company stop outsourcing and hire its own staff?

When several programs run concurrently, the work is consistent, and you can provide sufficient support for at least two or three programmers with adequate standards and quality control processes. Before reaching this level, outsourcing or a hybrid approach typically represents less risk.

Talk to Weltrix

If you are debating whether to pursue in-house or outsourced TLF programming for your next study, a brief conversation can save you weeks of planning. Contact Weltrix with your study phase and data cut dates, and we will tell you what is required and whether we are the right fit for your needs.