Welcome to the Logic Definition Forecasting Course. In this course, we will build upon the information covered in the CDSi Logic
Specification Overview, Supporting Data Tables, and Evaluation Courses. Specifically, we will provide additional detail on the topic of
Forecasting of target dose dates.
In this section we will briefly look at the purpose of the course, the topics we will cover and how this course fits with other training opportunities.
The goal of this training series is to enable the use of the CDSi Logic Specification in your environment. This course will contribute by
providing a basic overview of the Logic Definition for Forecasting, some remaining pieces that are in the Logic Definition, including Select Best
Patient Series, as well as Identify and Evaluate the Vaccine Group, and the third component in the CDSi Logic Specification, which is the
Processing Model. The training materials were produced for a diverse audience ranging from high level Program
Managers to Business Analysts to more detailed technical Architects and Developers with a variety of interests and knowledge regarding both clinical
and IT aspects of immunization evaluation and forecasting. In addition, we expect participants from the IIS community, EHR community, and
HIE communities. The emphasis in this course is on concepts that may be applied differently in a variety of local
implementations. It will not address one particular technical implementation, but rather the concepts that are applicable amongst all of them.
We are assuming a basic understanding of immunization evaluation and forecasting. However, we will lay a foundation to ensure all
audiences are effectively supported. This course has been design based on the assumption that the prior courses in the series
have been attended. We will interject a few brief quizzes during this presentation to checkpoint your understanding.
This course is designed to be standalone; however, there are additional resources that can help with the understanding of the concepts.
The General Recommendations Table 1, which documents ACIP's recommendations, is available in both PDF and html format.
The Logic Specification is available via the CDSi website. Within the Logic Specification, Appendix B
provides Acronyms and Abbreviations of some common terms. And for additional insight into some of the basics
around immunization evaluation and forecasting, the CDC Education, Information and Partnership Branch, the EIPB, has web-based courses.
Two in particular that would provide additional insight and be useful are: "Immunization: You Call the Shots" and "Epidemiology & Prevention of Vaccine-Preventable Diseases."
The topics covered in this course include a complete look at the Logic Definition, from start to finish, including how all the components work
together from Evaluation at the antigen level through providing a vaccine group forecast. Then we will look at concepts around
Forecasting. We will progress into Select Best Series, and we will round out the entire Logic Definition by briefly
discussing Vaccine Group and Processing Model concepts. We will finish by looking at additional sources of
assistance as you work to understand the CDSi Logic Specification.
This course continues where the Evaluation Course left off, providing additional detail specifically in the area of Forecasting.
In this section an overview of the Logic Definition will be provided from Evaluation all the way through Identifying and Evaluating Vaccine Groups
As covered in the Overview Course, the Logic Specification has three key components - the Supporting Data, the Logic Definition and the
Processing Model. The Supporting Data is the portion of the Specification that varies depending on the
antigen. It can be thought of as configuration data that will need to be updated as the recommendations change or to handle state-
specific and local differences. An example of this would be 'the absolute minimum age for Varicella versus the absolute minimum age for PCV.'
In summary, the Supporting Data helps to isolate the portion of the Logic Specification that must change more frequently while keeping the rest as
stable as possible. The Logic Definition describes the logical functionality necessary to evaluate shots that
have been given and determine Target Dose Dates which specify when shots should be given. The Processing Model contains more technical
details necessary to bring the patient's Immunization History together, with the supporting data to be processed by the Logic
Definition. Remember, the artifacts used to document the Logic Definition, which we will see in more detail
in this course, are: very high level process models that give a very basic overview of and provide context for the particular topic; decision
tables which concisely and very specifically provide a set of outcomes based on a set of incoming conditions; business rules, and other
descriptions and illustrations to help with the understanding of the Logic Definition.
There are four components of the Logic Definition that actually work together to process the patient's history using the Supporting Data.
Evaluation evaluates the patient's Immunization History against a specific path toward immunity. The focus of this course, Forecasting, which
based on the evaluation, determines when the patient's next shot should occur. Select Best Patient Series, which determines of
all the possible paths to immunity, which path is the best one given the set of circumstances present.
And Evaluate Vaccine Group which brings all of the antigen-level recommendations back into a vaccine-based forecast.
In the context of the Logic Specification, the Supporting Data, Logic Definition and Processing Model work together to process a patient's
immunization history, to evaluate their progress along the various paths to immunity and to forecast the best recommended dates for vaccinations.
As we mentioned, the Logic Definition is composed of four components, which cover the spectrum of Evaluation through Forecasting of
vaccine group administration dates.
As illustrated here, ultimately the forecast needs to be provided at the vaccine group level. For example, MMR.
As mentioned in the Evaluation Course, we simplify the entire process by focusing on one component at a time. In this case, the forecast
will first be determined on the antigen level. Thus, for MMR, we would first solve for Measles, Mumps and Rubella separately.
In many cases there are multiple paths to immunity.; therefore, the focus is further restricted to evaluating and forecasting dates for
one patient series at a time. Then all patient series within an antigen can be compared to determine the best series for that
antigen. And finally, all best series within a vaccine group can be combined into a vaccine group-level
forecast. Let's take the example of HepB. One path to immunity is a 3-dose series;
therefore evaluation and forecasting must be done based on that series. However, we also must perform evaluation and
forecasting on the other 4 series, including the 4-dose standard, 4-dose Comvax, 2-dose Adolescent and 3-dose Twinrix series.
At this point, the patient's Immunization History has been completely evaluated against the possible paths to immunity for Hep B.
Now, the evaluation and forecasting results for each path to immunity within an antigen must be compared to determine the Best Patient Series to
proceed with. In the case of HepB, there are five paths to compare. In the case of Measles there is no comparison, simply a fall through, because
there is only one possible path.
Now, the best series from each antigen must be combined into the proper vaccine group forecast. For example, the best series for Measles,
Mumps, and Rubella must be brought together into one Vaccine Group Level, or MMR in this case, forecast. In the case of HepB this is simply
a fall through, because there is only one antigen. Now that we have seen the entire process, we will step back through each aspect of the Logic
Definition in more detail.
Before we continue, please take this brief quiz.
As covered in the Evaluation Course, the goal of Evaluation is to determine the patient's status along the path to immunity for an antigen.
The inputs are the patient's Immunization History along with the Supporting Data for the particular antigen series being evaluated against.
Once we perform the evaluation, for each vaccine dose that was administered in the patient's history, we have a status of 'valid' or 'not valid'
as compared to a particular Target Dose that we are trying to satisfy in the path to immunity. The Patient Series is the resulting
combination of evaluated patient doses administered and target doses along with the overall series status. In this example, the Patient
Series is not complete because there is one target dose that needs to be satisfied.
In this section, we will discuss Forecasting. The goal of Forecasting is to determine dates based on the outputs of the evaluation of a
specific path for an antigen. We are not looking at all possibilities, only one specific path at a time. Ultimately, a forecast will be produced for each
antigen series. The Inputs are the patient's Immunization History, the Antigen Series Data from the Supporting Data
and the Evaluated Patient Series, which was the output of Evaluation. In this example, the forecast will result in an
Earliest, Recommended, Past Due and Latest Date for the 1 remaining Target Dose in the evaluated patient series.
Given forecasting is based on the evaluation for just one antigen at a time, the process for determining the dates is relatively straight
forward. The heavy lifting all occurred during the evaluation of the Patient series.
The first three sub-tasks of forecasting are checks to determine if the forecast needs to be calculated. In order to continue, we need to
answer the questions: Can a Target Dose be skipped because the person has passed some trigger age and the dose is no longer needed?
Can the Target Dose be substituted for another antigen and target dose? Should the patient receive another dose?
This is similar to the screening a doctor would perform prior to administering a vaccine. This includes checks like contraindications, immunity,
and checking whether the patient is past the maximum age. If there is a need to forecast and it has been
determined that we cannot skip or substitute doses, then the next step in the process is to generate the forecasted dates.
There are two types of forecasted dates. Unadjusted Forecast Dates are based strictly on the standard preferred schedule pulled straight
from the ACIP tables. They are frequently used for determining metrics or to help provide information to compare a patient to the standard
schedule. Adjusted Forecast Dates take into account the timeliness of the patient's immunization history
and represent the recommendation given the specific needs and situation of the patient as guided by the ACIP Recommendations.
The Earliest Date, represents the minimum gap after which vaccine administration can occur. It is determined by selecting the latest of: the
Minimum Age Date, Latest Minimum Interval Date, Latest Conflict End Date, and Seasonal Recommendation Start Date.
If we do not choose the latest of these dates, then the shot would be given at a point that is in conflict with a date requirement, thus resulting in
not valid Vaccine Dose Administered. The standard 4-day grace period is not taken into account when forecasting recommended dates.
It instead comes into play when a patient's history of administrations is being evaluated.
Before we continue, please take this brief quiz.
The Adjusted Recommended Date is the later of the Earliest Forecast Date, which we just discussed, or the Earliest Recommended Age
Date. If the Earliest Recommended Age is not provided the Earliest Recommended Interval is used instead.
Again, this comparison ensures that recommendations are not provided which are prior to the earliest date that an administered
dose can be considered valid.
The Past Due Date is similarly adjusted to ensure that it occurs after the earliest date a dose can be considered valid.
The Latest Recommended Age Date is adjusted down by one day to ensure consistent comparisons are made. This is the last day
through which the patient is not past due. After this date, they are past due.
The Latest Date, if noted, represents the last day someone can get a shot. For example, it is 7 years on DTaP.
Given the bulk of the work was done in Evaluation, forecasting has simply become a matter of comparing possible dates to determine
the Earliest and Recommended Dates.
In this section we will look at Select Best Patient Series.
Based on the various paths to immunity for a particular antigen, we need to determine the one that will best fit the need.
In addition to the patient's information and supporting data, the results of Evaluation and Forecasting for every antigen series within a
particular antigen are used by Select Best Patient Series. Once processed, the resulting output is the best
patient series, including the forecasted Target Dose dates for that series.
There are four steps to ultimately select the Best Patient Series. First, a clearly superior patient series is looked
for. If there is one that is clearly superior, it is selected. If not, the Patient Series are compared to determine which ones are worth bringing
forward and how they will be scored. Now, the Patient Series brought forth are scored with the previously selected rules.
And finally, the scores are evaluated and the best, or highest scoring, Patient Series is selected.
A general principle for the Logic Specification is that when deciding what to choose or where to spend energy, 'complete' is considered better
than 'in-process', which is considered better than 'not started' at all. 'Complete' means that all appropriate target
doses in a patient series have been satisfied or appropriately skipped. In these cases, the Patient Series will have a status of 'completed'.
'In-process' means that one or more target doses have been satisfied, but not all. In this illustration, the 3rd target dose has not been
satisfied yet; therefore the Patient Series is 'in- process.' Above it shows a complete series, in which both
Target Doses have been satisfied. Clearly the complete series is better because based on that path the patient already has immunity; therefore,
it is not necessary to have the patient come back for more shots, as would be required for the 'in- process' series.
Likewise, an in-process series is better than a series in which none of the Target Doses that were administered were valid for satisfying any of
the Target Doses.
In order to determine if there is clearly one superior Patient Series, the following is considered.
If there is only one Patient Series, then that is the superior series. Given our principle of more complete being
preferred, if only one of the series being considered is complete then that is the superior series.
If only one of the series is in-process and none are complete, then that is the superior series. If all considered series are 'not-started', but one
is identified in the Supporting Data as the default series, then that is the superior series. Based on the above, if we still cannot select a
superior Patient Series then we need to with the standard process, which includes classifying, scoring and selecting the Best Patient Series
There are three sets of rules for scoring patient series: rules for completed series, rules for in- process series and rules for not-started series.
The rules have been divided into these groups to simplify the comparisons being made. Only Patient Series with the same relative
progress, either complete, in-process or not- started are considered. The other series are dropped.
The choice of which rule set and series to consider is based on the principle discussion earlier. Complete is better than in-process which
is better than not started; therefore, in our example, only the top two series would be considered and the in-process rules would apply.
This is due to the fact that the bottom series has no satisfied target doses; therefore, it is not started.
The top two series each have one target dose that has not been satisfied; therefore, they are both in-process.
Before we continue, please take this brief quiz.
Scoring uses a relative point system in which points are either added or deducted from the series.
The scoring is based on a variety of conditions. In some cases, points are added if the condition being evaluated is true for only one of the Patient
Series that is being compared. For example, if comparing three series and the condition is only true for one of them, then a point is added only for
that one. Sometimes points are added if the condition is true for two or more Patient Series.
Points can also be deducted if the condition is not true for the particular patient series that is being scored.
The scoring point system was group-developed. It started with what the subject matter experts found to be the most valuable to judge the series,
such as 'finishing as early as possible', or 'trying to have the least number of invalid doses.' Then the group incorporated concepts already
implemented in two existing CDS engines. The scoring point system was then evolved through a series of trial and error, test cases,
analysis, peer discussion and guidance from the EIPB Liaison.
Here is an example of the scoring rules for a complete Patient Series. The conditions being evaluated appear in the first
column. If the condition is only true for the Patient Series being considered, then the patient series' score is
adjusted by the values in the second column. If the condition is true for the Patient Series being considered, as well as other patient series, the
score is adjusted by the values in the third column. If the condition is not true for the Patient Series
being considered, the score is adjusted down by the values in the last column. Once all Patient Series being considered have
been scored, the points are tallied to choose the highest scoring series.
We have now selected the Best Patient Series for each antigen. Now the antigen level forecast must be brought back together into a vaccine
group level forecast.
In most cases, combining the antigen-level paths into vaccine group level paths is relatively straight forward, as there is a single antigen per vaccine
group. There are a handful of cases, including MMR, DTaP, Tdap, and Td which are more complicated because they each contain multiple
antigens. When completed, we will end up with a Vaccine Group Forecast that provides the forecasted
dates for a given Vaccine Group. Given the limited usage of Identify and Evaluate Vaccine Group, further details concerning this
topic are beyond the scope of this training. Additional details can be found in the Logic Specification.
To summarize, we started by evaluating and forecasting per antigen series. Then we selected the Best Patient Series per
antigen. And finally, the best antigen series were pulled together into a Vaccine Group Forecast.
In this section we are going to briefly look at the Processing Model which provides the final details necessary to pull all the pieces of the Logic
Specification together. The Processing Model presented in the CDSi Logic Specification is just one of many possible
approaches to implementing the functionality in the Logic Definition against the patient's immunization history, using the supporting data.
The high-level process illustrated here first prepares all of the data needed, then steps through evaluation and forecasting for each
Patient Series. Next the Best Patient Series is chosen per antigen and finally brought back together into the appropriate vaccine groups.
More detailed and algorithmic models for each of these 6 activities is available in the processing model section of the CDSi Logic Specification.
Again, your local implementation of the processing model may be implemented differently.
Please contact us if you need more help or guidance in this area.
Now that we have completed our coverage of the Logic Specification, we will briefly look at additional sources of assistance
As mentioned in the Overview Course, NIP Info is now available to take inquiries around the Logic Specification and Testing Methodologies. If your
inquiry is specific to the Logic Specification or Testing Methodology, as opposed to the ACIP recommendations, be sure to clearly state this
early in your message. This is to help ensure the CDSi support team is appropriately included as necessary.
From the CDSi website, you have access to the current versions of the Logic Specification, including supporting data and the testing
methodology, including test scripts and the test script user guide. In addition, training materials, FAQs, and other
support documents are provided. There will continue to be periodic communications from the CDSi support team.
And as implementations occur, there will be Q&A Webinars.
This concludes the series of courses covering the various aspects of the Logic Specification. Additional detail on the Test Cases and Test Case
User Guide are provided in the Testing Methodology course.