Semetko & Valkenburg generic news frames
The five generic news frames of Semetko & Valkenburg (2000), measured with yes/no questions per story: resolution (who can fix the problem, and how), human interest, conflict, morality, and economic consequences. It reports how often each frame appears, overall and by outlet, group, or period, with the questions behind each one. The questions and coding rules are those Frermann et al. (2023) gave their coders, which a confirmatory factor analysis showed still form the five frames.
The program
1 steps- 1
framesjev Jev · measuring framesTwenty-one yes/no questions about the story as a whole, each with the coders' rule and, where the codebook has them, examples.
→ writesreadings
Stages
22 questionsframes · 22 questions
jev-1.13.0Twenty-one yes/no questions about the story as a whole, each with the coders' rule and, where the codebook has them, examples.
lens semetko_valkenburg_frames· the whole text is given as article
As measured, the yes/no criteria of 9 questions were not sent to Jev (hi_face, hi_benefit, co_disagree, co_reproach, co_sides, co_winners, ec_financial, ec_costs, ec_consequences): the card reproduces that so its track record holds.
The questions
as Jev is asked themyes / no Suggests a solution re_solution
[object Object]
yes / no Problem resolved re_resolved
[object Object]
yes / no Hope of resolution re_hope
[object Object]
yes / no Urgent action needed re_urgent
[object Object]
yes / no Hero: some entity could alleviate the problem re_alleviate
[object Object]
yes / no Villain: some entity is responsible for the problem re_responsible
[object Object]
yes / no A human face on the issue hi_face
[object Object]
- yes
- —
- no
- —
yes / no Emotive language hi_emotion
[object Object]
yes / no Victim: some entity is harmed by the problem hi_harmed
[object Object]
yes / no Someone benefits hi_benefit
[object Object]
- yes
- —
- no
- —
yes / no Private lives hi_private
[object Object]
yes / no Disagreement co_disagree
[object Object]
- yes
- —
- no
- —
yes / no Reproach co_reproach
[object Object]
- yes
- —
- no
- —
yes / no Two or more sides co_sides
[object Object]
- yes
- —
- no
- —
yes / no Winners and losers co_winners
[object Object]
- yes
- —
- no
- —
yes / no Moral message mo_message
[object Object]
choice mo_whose mo_whose
[object Object]
own- The article itself judges right and wrong in its own voice: it condemns someone's selfishness or hypocrisy, praises people for doing the right thing, or tells the reader what is right
quoted- Moral judgments appear only in what the article quotes or reports others saying ("Kerry called the withdrawal shameful"); the article itself does not take them up
none- No one in the article judges anything as morally right or wrong
yes / no Religion or moral obligation mo_religion
[object Object]
yes / no Prescribes how to behave mo_prescribe
[object Object]
yes / no Financial losses or gains ec_financial
[object Object]
- yes
- —
- no
- —
yes / no Costs or amounts ec_costs
[object Object]
- yes
- —
- no
- —
yes / no Economic consequences of acting or not ec_consequences
[object Object]
- yes
- —
- no
- —
What it reports
32 indicators| Indicator | Kind | From | Needs |
|---|---|---|---|
| Resolution frameheadline | frame | re_solutionre_alleviate | 2 of 2 yes |
| Human interest frameheadline runs high (30% of climate articles where the coders found 13%, with or without calibration): the model's answers to its three questions go together more often than the coders' do | frame | hi_facehi_emotionhi_private | 2 of 3 yes |
| Conflict frameheadline | frame | co_disagreeco_reproachco_sides | 2 of 3 yes |
| Morality frameheadline | frame | mo_whose:ownmo_religion | 1 of 2 yes |
| Economic consequences frameheadline | frame | ec_financialec_costsec_consequences | 2 of 3 yes |
| Resolution scale | scale | re_solutionre_alleviate | – |
| Human interest scale | scale | hi_facehi_emotionhi_private | – |
| Conflict scale | scale | co_disagreeco_reproachco_sides | – |
| Morality scale | scale | mo_messagemo_religion | – |
| Economic consequences scale | scale | ec_financialec_costsec_consequences | – |
| Hero: some entity could alleviate the problem | share | re_alleviate | – |
| Villain: some entity is responsible for the problem | share | re_responsible | – |
| Victim: some entity is harmed by the problem | share | hi_harmed | – |
| Suggests a solution | share | re_solution | – |
| Problem resolved | share | re_resolved | – |
| Hope of resolution | share | re_hope | – |
| Urgent action needed | share | re_urgent | – |
| A human face on the issue | share | hi_face | – |
| Emotive language | share | hi_emotion | – |
| Someone benefits | share | hi_benefit | – |
| Private lives | share | hi_private | – |
| Disagreement | share | co_disagree | – |
| Reproach | share | co_reproach | – |
| Two or more sides | share | co_sides | – |
| Winners and losers | share | co_winners | – |
| Moral message counts quoted judgments of right and wrong (65% against the coders' 12%) | share | mo_message | – |
| Moral judgment in the article's own voice | share | mo_whose:own | – |
| Religion or moral obligation | share | mo_religion | – |
| Prescribes how to behave | share | mo_prescribe | – |
| Financial losses or gains | share | ec_financial | – |
| Costs or amounts | share | ec_costs | – |
| Economic consequences of acting or not | share | ec_consequences | – |
Calibration
Standard: the Narrative Frames Corpus coders (Frermann et al. 2023)
Fit on: the paper's folds 1-2 (171 articles): per question, the threshold (0.05-0.95) maximizing mean kappa with the individual coders; scored on folds 3-5
Source: project_research/experiments/framing/05_calibrate.py
Track record
against human coders and external anchorsQuestion by question, Jev agrees with a coder as often as the coders agree with each other: median Cohen's kappa 0.36 for both (0.41 vs 0.42 on the 13 frame-rule questions), at or above the coders on 12 of 21 questions; Krippendorff's alpha 0.52 among the coders and 0.51 with Jev added as a coder. Frames on the paper's five test folds, zero-shot: macro F1 0.62, against 0.61 for the paper's best supervised model (precision 0.47, recall 0.89). Resolution, conflict, and economic frames within 0.07 F1 of a held-out coder; morality far too frequent (66% vs 10%). Resolution more common in left than right outlets, as published (-13 points, 95% CI -22 to -4; published -16).
source: project_research/experiments/framing/RESULTS.md
With the morality forced choice and the calibrated thresholds: frame macro F1 0.69 on folds 3-5 (0.62 in round 1 on the same folds); Jev at or above the coders' kappa on 15 of 21 questions (9 at 0.5); better than a held-out coder on resolution, conflict, human interest, and morality, worse on economic (0.62 vs 0.70). Morality frame 18% of articles against the coders' 11%; human interest still 30% against 13%.
source: project_research/experiments/framing/RESULTS.md (round 2)
Rules and limitations
- The model sees the story's text only. Outlet, group, date, and author are joined afterwards, as the coders worked blind to them.
- A frame is present when enough of its questions are yes (two; one for morality), the rule Frermann et al. (2023) validated with a confirmatory factor analysis. A question counts as yes above its calibrated threshold (the calibration block), or above 0.5 with compile(..., calibrated=False). Scales and shares average the probabilities and are not calibrated.
- Every question carries the coders' reading: code what is explicit, follow the writer's intention, judge the main focus (title and opening), and answer no when unsure.
- The frames are generic: they say how a story is told, not what it is about. Issue-specific frames need their own questions.
- Validated so far on one issue (climate change) in US and UK online news, 2017-2019.
- The model says yes more readily than trained coders. Calibration to the Narrative Frames coders narrows it (precision 0.57, recall 0.85 on held-out folds) but frames are still found in more stories than those coders found. Compare groups and periods with each other, not levels with another study's.
- The calibration fits one team's coding standard on one issue. Another team's coders, or another issue, may draw the line elsewhere; compile with calibrated=False to see the uncalibrated levels beside it.
- Morality: in version 0.1 the frame used the moral-message question and found a moral frame in 66% of climate articles where the coders found 10%, because the model counted judgments the article only quotes. Since 0.2 it uses a forced choice between the article's own judgment, a quoted one, and none.
- English stories only. Visual content (Semetko & Valkenburg's tenth question) is not coded.