MOVIEMOD: An Implementable Decision-Support System for Prerelease Market Evaluation of Motion Pictures

TLDR: In this article, the authors present an interactive Markov chain model for predicting the box-office performance of motion pictures based on a behavioral representation of the consumer adoption process for movies as a macroflow process.

Abstract: In spite of the high financial stakes involved in marketing new motion pictures, marketing science models have not been applied to the prerelease market evaluation of motion pictures. The motion picture industry poses some unique challenges. For example, the consumer adoption process for movies is very sensitive to word-of-mouth interactions, which are difficult to measure and predict before the movie has been released. In this article, we undertake the challenge to develop and implement MOVIEMOD-a prerelease market evaluation model for the motion picture industry. MOVIEMOD is designed to generate box-office forecasts and to support marketing decisions for a new movie after the movie has been produced or when it is available in a rough cut but before it has been released. Unlike other forecasting models for motion pictures, the calibration of MOVIEMOD does not require any actual sales data. Also, the data collection time for a product with a limited lifetime such as a movie should not take too long. For MOVIEMOD it takes only three hours in a "consumer clinic" to collect the data needed for the prediction of box-office sales and the evaluation of alternative marketing plans. The model is based on a behavioral representation of the consumer adoption process for movies as a macroflow process. The heart of MOVIEMOD is an interactive Markov chain model describing the macro-flow process. According to this model, at any point in time with respect to the movie under study, a consumer can be found in one of the following behavioral states: undecided, considerer, rejecter, positive spreader, negative spreader, and inactive. The progression of consumers through the behavioral states depends on a set of movie-specific factors that are related to the marketing mix, as well as on a set of more general behavioral factors that characterize the movie-going behavior in the population of interest. This interactive Markov chain model allows us to account for word-of-mouth interactions among potential adopters and several types of word-of-mouth spreaders in the population. Marketing variables that influence the transitions among the states are movie theme acceptability, promotion strategy, distribution strategy, and the movie experience. The model is calibrated in a consumer clinic experiment. Respondents fill out a questionnaire with general items related to their movie-going and movie communication behavior, they are exposed to different sets of information stimuli, they are actually shown the movie, and finally, they fill outpostmovie evaluations, including word-of-mouth intentions.These measures are used to estimate the word-of-mouth parameters and other behavioral factors, as well as the movie-specific parameters of the model. MOVIEMOD produces forecasts of the awareness, adoption intention, and cumulative penetration for a new movie within the population of interest for a given base marketing plan. It also provides diagnostic information on the likely impact of alternative marketing plans on the commercial performance of a new movie. We describe two applications of MOVIEMOD: One is a pilot study conducted without studio cooperation in the United States, and the other is a full-fledged implementation conducted with cooperation of the movie's distributor and exhibitor in the Netherlands. The implementations suggest that MOVIEMOD produces reasonably accurate forecasts of box-office performance. More importantly, the model offers the opportunity to simulate the effects of alternative marketing plans. In the Dutch application, the effects of extra advertising, extra magazine articles, extra TV commercials, and higher trailer intensity compared to the base marketing plan of the distributor were analyzed. We demonstrate the value of these decision-support capabilities of MOVIEMOD in assisting managers to identify a final plan that resulted in an almost 50% increase in the test movie's revenue performance, compared to the marketing plan initially contemplated. Management implemented this recommended plan, which resulted in box-office sales that were within 5% of the MOVIEMOD prediction. MOVIEMOD was also tested against several benchmark models, and its prediction was better in all cases. An evaluation of MOVIEMOD jointly by the Dutch exhibitor and the distributor showed that both parties were positive about and appreciated its performance as a decision-support tool. In particular, the distributor, who has more stakes in the domestic performance of its movies, showed a great interest in using MOVIEMOD for subsequent evaluations of new movies prior to their release. Based on such evaluations and the initial validation results, MOVIEMOD can fruitfully and inexpensively be used to provide researchers and managers with a deeper understanding of the factors that drive audience response to new motion pictures, and it can be instrumental in developing other decision-support systems that can improve the odds of commercial success of new experiential products.

Full text

'83@/<=3>C90#/88=C6@+83+'83@/<=3>C90#/88=C6@+83+
%-296+<6C97798=%-296+<6C97798=
":/<+>398=809<7+>398+8./-3=398=#+:/<= )2+<>98+-?6>C$/=/+<-2

"( "87:6/7/8>+,6//-3=398%?::9<>%C=>/709< "( "87:6/7/8>+,6//-3=398%?::9<>%C=>/709<
#</</6/+=/ +<5/>@+6?+>39890 9>398#3->?</=#</</6/+=/ +<5/>@+6?+>39890 9>398#3->?</=
/29=2?+63+=2,/<1
'83@/<=3>C90#/88=C6@+83+
/.3.+2985/<
92+8,3<%%+A28/C
9669A>23=+8.+..3>398+6A9<5=+>2>>:=</:9=3>9<C?:/88/.?93.*:+:/<=
#+<>90>2/ +<5/>38197798=">2/<%9-3+6+8./2+@39<+6%-3/8-/=97798=+8.>2/$/-</+>398
?=38/==97798=
$/-977/8./.3>+>398$/-977/8./.3>+>398
63+=2,/<1985/<%+A28/C % "( "87:6/7/8>+,6//-3=398%?::9<>
%C=>/709<#</</6/+=/ +<5/>@+6?+>39890 9>398#3->?</=
+<5/>381%-3/8-/



2>>:.B.939<175=-
&23=:+:/<3=:9=>/.+>%-296+<6C97798=2>>:=</:9=3>9<C?:/88/.?93.*:+:/<=
9<79</3809<7+>398:6/+=/-98>+-></:9=3>9<C:9,9B?:/88/.?

"( "87:6/7/8>+,6//-3=398%?::9<>%C=>/709<#</</6/+=/ +<5/> "( "87:6/7/8>+,6//-3=398%?::9<>%C=>/709<#</</6/+=/ +<5/>
@+6?+>39890 9>398#3->?</=@+6?+>39890 9>398#3->?</=
,=><+->,=><+->
8=:3>/90>2/2312I8+8-3+6=>+5/=38@96@/.387+<5/>3818/A79>398:3->?</=7+<5/>381=-3/8-/79./6=
2+@/89>,//8+::63/.>9>2/
:</</6/+=/
7+<5/>/@+6?+>3989079>398:3->?</=&2/79>398:3->?</38.?=><C
:9=/==97/?83;?/-2+66/81/=9</B+7:6/>2/-98=?7/<+.9:>398:<9-/==09<79@3/=3=@/<C=/8=3>3@/
>9A9<.9079?>238>/<+->398=A23-2+</.3K-?6>>97/+=?</+8.:</.3->
,/09</
>2/79@3/2+=,//8
</6/+=/.8>23=+<>3-6/A/?8./<>+5/>2/-2+66/81/>9./@/69:+8.37:6/7/8> "( "F+:</</6/+=/
7+<5/>/@+6?+>39879./609<>2/79>398:3->?</38.?=><C "( "3=./=318/.>91/8/<+>/,9B9K-/
09</-+=>=+8.>9=?::9<>7+<5/>381./-3=398=09<+8/A79@3/+0>/<>2/79@3/2+=,//8:<9.?-/.9<
A2/83>3=+@+36+,6/38+<9?12-?>,?>,/09</3>2+=,//8</6/+=/.'8635/9>2/<09</-+=>38179./6=09<
79>398:3->?</=>2/-+63,<+>39890 "( ".9/=89></;?3</+8C+->?+6=+6/=.+>+6=9>2/.+>+
-966/->398>37/09<+:<9.?->A3>2+6373>/.630/>37/=?-2+=+79@3/=29?6.89>>+5/>9969819<
"( "3>>+5/=986C>2<//29?<=38+G-98=?7/<-6383-H>9-966/->>2/.+>+8//./.09<>2/:</.3->39890
,9B9K-/=+6/=+8.>2//@+6?+>39890+6>/<8+>3@/7+<5/>381:6+8=
&2/79./63=,+=/.98+,/2+@39<+6</:</=/8>+>39890>2/-98=?7/<+.9:>398:<9-/==09<79@3/=+=+
7+-<9J9A:<9-/==&2/2/+<>90 "( "3=+838>/<+->3@/ +<59@-2+3879./6./=-<3,381>2/7+-<9
J9A:<9-/==--9<.381>9>23=79./6+>+8C:938>38>37/A3>2</=:/->>9>2/79@3/?8./<=>?.C+
-98=?7/<-+8,/09?8.3898/90>2/09669A381,/2+@39<+6=>+>/=?8./-3./.-98=3./</<</4/->/<:9=3>3@/
=:</+./<8/1+>3@/=:</+./<+8.38+->3@/&2/:<91</==39890-98=?7/<=>2<9?12>2/,/2+@39<+6=>+>/=
./:/8.=98+=/>90
79@3/=:/-3I-
0+->9<=>2+>+</</6+>/.>9>2/7+<5/>38173B+=A/66+=98+=/>90
79</1/8/<+6
,/2+@39<+6
0+->9<=>2+>-2+<+->/<3D/>2/79@3/19381,/2+@39<38>2/:9:?6+>3989038>/</=>
&23=38>/<+->3@/ +<59@-2+3879./6+669A=?=>9+--9?8>09<A9<.9079?>238>/<+->398=+7981:9>/8>3+6
+.9:>/<=+8.=/@/<+6>C:/=90A9<.9079?>2=:</+./<=38>2/:9:?6+>398 +<5/>381@+<3+,6/=>2+>
38J?/8-/>2/><+8=3>398=+7981>2/=>+>/=+</79@3/>2/7/+--/:>+,363>C:<979>398=><+>/1C.3=><3,?>398
=><+>/1C+8.>2/79@3//B:/<3/8-/&2/79./63=-+63,<+>/.38+-98=?7/<-6383-/B:/<37/8>
$/=:98./8>=I669?>+;?/=>3988+3</A3>21/8/<+63>/7=</6+>/.>9>2/3<79@3/19381+8.79@3/
-977?83-+>398,/2+@39<>2/C+<//B:9=/.>9.300/</8>=/>=903809<7+>398=>37?63>2/C+</+->?+66C
=29A8>2/79@3/+8.I8+66C>2/CI669?>:9=>79@3//@+6?+>398=38-6?.381A9<.9079?>238>/8>398=&2/=/
7/+=?</=+</?=/.>9/=>37+>/>2/A9<.9079?>2:+<+7/>/<=+8.9>2/<,/2+@39<+60+->9<=+=A/66+=>2/
79@3/=:/-3I-:+<+7/>/<=90>2/79./6
"( ":<9.?-/=09</-+=>=90>2/+A+</8/==+.9:>39838>/8>398+8.-?7?6+>3@/:/8/><+>39809<+
8/A79@3/A3>238>2/:9:?6+>3989038>/</=>09<+13@/8,+=/7+<5/>381:6+8>+6=9:<9@3./=.3+189=>3-
3809<7+>39898>2/635/6C37:+->90+6>/<8+>3@/7+<5/>381:6+8=98>2/-977/<-3+6:/<09<7+8-/90+8/A
79@3/)/./=-<3,/>A9+::63-+>398=90 "( ""8/3=+:369>=>?.C-98.?->/.A3>29?>=>?.39
-99:/<+>39838>2/'83>/.%>+>/=+8.>2/9>2/<3=+0?66J/.1/.37:6/7/8>+>398-98.?->/.A3>2
-99:/<+>39890>2/79@3/=.3=><3,?>9<+8./B23,3>9<38>2/!/>2/<6+8.=&2/37:6/7/8>+>398==?11/=>>2+>
"( ":<9.?-/=</+=98+,6C+--?<+>/09</-+=>=90,9B9K-/:/<09<7+8-/ 9</37:9<>+8>6C>2/
79./6900/<=>2/9::9<>?83>C>9=37?6+>/>2//00/->=90+6>/<8+>3@/7+<5/>381:6+8=8>2/?>-2
+::63-+>398>2//00/->=90/B><++.@/<>3=381/B><+7+1+D38/+<>3-6/=/B><+&(-977/<-3+6=+8.2312/<
><+36/<38>/8=3>C-97:+</.>9>2/,+=/7+<5/>381:6+890>2/.3=><3,?>9<A/</+8+6CD/.)/./798=><+>/
>2/@+6?/90>2/=/./-3=398=?::9<>-+:+,363>3/=90 "( "38+==3=>3817+8+1/<=>93./8>30C+I8+6
:6+8>2+></=?6>/.38+8+679=>38-</+=/38>2/>/=>79@3/=</@/8?/:/<09<7+8-/-97:+</.>9>2/
7+<5/>381:6+8383>3+66C-98>/7:6+>/. +8+1/7/8>37:6/7/8>/.>23=</-977/8./.:6+8A23-2</=?6>/.
38,9B9K-/=+6/=>2+>A/</A3>23890>2/ "( ":</.3->398 "( "A+=+6=9>/=>/.+1+38=>
=/@/<+6,/8-27+<579./6=+8.3>=:</.3->398A+=,/>>/<38+66-+=/=

8/@+6?+>39890 "( "4938>6C,C>2/?>-2/B23,3>9<+8.>2/.3=><3,?>9<=29A/.>2+>,9>2:+<>3/=
A/</:9=3>3@/+,9?>+8.+::</-3+>/.3>=:/<09<7+8-/+=+./-3=398=?::9<>>9968:+<>3-?6+<>2/
.3=><3,?>9<A292+=79</=>+5/=38>2/.97/=>3-:/<09<7+8-/903>=79@3/==29A/.+1</+>38>/</=>38
?=381 "( "09<=?,=/;?/8>/@+6?+>398=908/A79@3/=:<39<>9>2/3<</6/+=/+=/.98=?-2
/@+6?+>398=+8.>2/383>3+6@+63.+>398</=?6>= "( "-+80<?3>0?66C+8.38/B:/8=3@/6C,/?=/.>9
:<9@3./</=/+<-2/<=+8.7+8+1/<=A3>2+.//:/<?8./<=>+8.38190>2/0+->9<=>2+>.<3@/+?.3/8-/
</=:98=/>98/A79>398:3->?</=+8.3>-+8,/38=><?7/8>+638./@/69:3819>2/<./-3=398=?::9<>
=C=>/7=>2+>-+837:<9@/>2/9..=90-977/<-3+6=?--/==908/A/B:/<3/8>3+6:<9.?->=
/CA9<.=/CA9<.=
79>398:3->?</=8/A:<9.?->=:</>/=>7+<5/>/@+6?+>39809</-+=>381./-3=398=?::9<> +<59@2+38=
3=-3:638/=3=-3:638/=
+<5/>381E">2/<%9-3+6+8./2+@39<+6%-3/8-/=E$/-</+>398?=38/==
&23=49?<8+6+<>3-6/3=+@+36+,6/+>%-296+<6C97798=2>>:=</:9=3>9<C?:/88/.?93.*:+:/<=

A-1
Appendix [to be posted on the Marketing Science web site]
More Detailed Information about the Implementation Described in the Paper:
MOVIEMOD: An Implementable Decision Support System for
Pre-Release Market Evaluation of Motion Pictures
Jehoshua Eliashberg
Jedid-Jah Jonker
Mohanbir S. Sawhney
Berend Wierenga
In this document we describe the procedures employed in implementing MOVIEMOD.
The document contains detailed information concerning measures, scales, model’s calibration,
sampling, and projection.
Measures and Scales
A questionnaire was used to obtain estimates for the various parameters of the
MOVIEMOD model. (It can be obtained from the authors upon request.) Both direct and indirect
measures were used. Below we provide an example of a question (and illustrative analysis) used
to obtain a direct measure, and an example of a question (and illustrative analysis) used to obtain
an indirect measure for word-of-mouth duration parameter (µ).
Direct measure:
On average, how many weeks do you continue to talk to friends about a movie
that you have recently seen at a theater?
a. 1 week or less
b. 2 to 3 weeks
c. 4 to 5 weeks
d. 6 weeks or more
Illustrative analysis:

A-2
Suppose 30% of the respondents indicated option (a), 30% option (b), 25% option (c),
and 15% option (d). An estimate for 1/µ would be: (30 Η 0.5 + 30 Η 2.5 + 25 Η 4.5 + 15 Η
7)/100 = 3.075. Because µ is a parameter characterizing a Poisson process (intensity per week),
the mean of the word-of-mouth duration process is equal to 1/µ (average duration in weeks).
Indirect measure:
Assume that you saw a movie this week that you liked. What are the chances
you would still be talking to your friends about the movie two weeks after
seeing the movie?
a. 90% change or more
b. 75% chance
c. 50% chance
d. 25% chance
e. 10% chance or less.
Illustrative analysis:
For the indirect measure we used equation (8) in the paper. First, we had to calculate the
average survival probability, p
k
(k = 2). Suppose 20% of the respondents indicated option (a),
15% option (b), 25% option (c), 30% option (d), and 10% option (e). The average survival
probability is then (20 Η 95 + 15 Η 75 + 25 Η 50 + 30 Η 25 + 10 Η 5)/100 = 50.75. Substituting
the estimated value for p
2
in equation (8), we obtained an estimate for µ.
Similar measures and analyses were used for the other behavioral paramters (δ, λ, and γ).
In the Dutch implementation, the number of measures employed were 2, 2, 4, and 1 for µ, δ, λ,
and γ, respectively. To assess the reliability of µ, δ, and λ, 8 pairwise Spearman’s correlation
coefficients were calculated (1 for µ, 1 for δ, and 6 for λ). Their average value was 0.273.
Model’s Calibration
Almost all of the parameters can be determined in a single-shot experiment. This is a very

Frequently asked questions

Q1. What are the contributions in "Moviemod: an implementable decision-support system for prerelease market evaluation of motion pictures" ?

In this article, the authors undertake the challenge to develop and implement MOVIEMOD—a prerelease market evaluation model for the motion picture industry. According to this model, at any point in time with respect to the movie under study, a consumer can be found in one of the following behavioral states: undecided, considerer, rejecter, positive spreader, negative spreader, and inactive. The authors describe two applications of MOVIEMOD: One is a pilot study conducted without studio cooperation in the United States, and the other is a full-fledged implementation conducted with cooperation of the movie 's distributor and exhibitor in the Netherlands. In the Dutch application, the effects of extra advertising, extra magazine articles, extra TV commercials, and higher trailer intensity ( compared to the base marketing plan of the distributor ) were analyzed. The authors demonstrate the value of these decision-support capabilities of MOVIEMOD in assisting managers to identify a final plan that resulted in an almost 50 % increase in the test movie 's revenue performance, compared to the marketing plan initially contemplated. Based on such evaluations and the initial validation results, MOVIEMOD can fruitfully ( and inexpensively ) be used to provide researchers and managers with a deeper understanding of the factors that drive audience response to new motion pictures, and it can be instrumental in developing other decision-support systems that can improve the odds of commercial success of new experiential products. This interactive Markov chain model allows us to account for word-of-mouth interactions among potential adopters and several types of word-of-mouth spreaders in the population. The implementations suggest that MOVIEMOD produces reasonably accurate forecasts of box-office performance. 

Q2. How many pairs of correlation coefficients were used to assess the reliability of, ?

To assess the reliability of µ, δ, and λ, 8 pairwise Spearman’s correlation coefficients were calculated (1 for µ, 1 for δ, and 6 for λ). 

Q3. How many times did the panel ask about their awareness of movies?

The R2 values obtained in the implementation described in the paper were 0.938 and0.768 in the U.S. and the Netherlands, respectively.