2019 International Conference on Social Computing, Behavioral-Cultural Modeling, & Prediction and Behavior Representation in Modeling and Simulation
July 09-12, 2019, Lehman Auditorium, George Washington University, Washington DC, USA

Disinformation is a growing problem on the web. Often this information is spread through multiple social media. While disinformation itself is not a new problem; it’s spread, the rate of spread, the potential for global impact and so on are growing due to the use of social media. Social media providers themselves are concerned and examining what can be done in this space.

In this year’s SBP-BRiMS second challenge problem, we ask participants to consider the issue of the spread of disinformation on the web.

The specific questions of interest are:

• Can we automatically and accurately classify a message as containing disinformation? And the related question, what are the characteristics of disinformation that make it distinct from other information? Eligible entries might compare against Truthy.
• What are the characteristics of individuals or groups that put them at risk to succumbing to disinformation? And the related question, how can you measure the extent to which an individual or group has succumbed to disinformation? Algorithm for measuring risk must be provided and validation strategy explained.
• How does disinformation spread within and across media? And the related question, does disinformation spread differently than other information? Information on how spread was measured must be explained.

These are the specific questions to be addressed. Each response may address one or more of these questions. All entries must have both a strong social theory, political theory or policy perspective and a strong methodology perspective.

• Participants may work individually or in teams.
• Participants must define what they mean by disinformation.
• Participants must use data related to at least two social media. Data from just on-line mainstream news sites is only of tertiary interest.
• Participants must address one of the three questions above.
• Participants should address a social science theory or policy relevant issue and should employ one or more methodologies appropriate for the empirical assessment of or forecasting on the basis of big data (e.g., computational algorithms, machine learning, computer simulation, social network analysis, text mining).
• Each participating team may prepare one or more entry.
• Entries must represent original work that has not been previously published or submitted to other challenges.
• Each participating team must send at least one member to the SBP-BRiMS 2019 conference to present a poster describing their entry, to do a presentatation that is video taped, and to, if they win, do a presentation on the last day. At least one team member must register and attend the conference.
• Participants are encouraged to use any data set of interest. However, this data must be provided along with the challenge paper. This data will become part of a public repository. To that end – a) the names of individuals in the data set (or their social media id), must be de-identified, b) the ids of specific social media posts should be replace by fake ids. If the submitting team does not know how to do that, Dr. Kathleen Carley will have a third party run a de-identifier on the data for the participating team.
• At the conference, all entries will be judged by the community using a participant voting system.
• The individual or group that submits the winning entry, and that submitting the runner-up entry, will have their full length paper describing their challenge solution published in a SBP-BRiMS special issue of the journal Computational and Mathematical Organization Theory. Submission to the challenge problem means that you consent to publish the fill length paper in this venue.

A strong entry will have one or more of these components:

• Employ multiple data sets.
• Include a high quality visualization (note that participants will be allowed to display dynamic visualizations via some form of electronic media e.g., by hanging a tablet from the poster. However, please note that tables will not be provided.
• Account for biases in the data across time, space, topics and sources.
• Provide a new metric or algorithm development such as:
1. A new spatial, temporal, and network analytic methodologies and algorithms that can cope with the vast scale of open source data (e.g. Twitter data) and support answering a key social or policy issue.
2. A new spatial analytic methodology that can better take into account change over time and non-spatial distances (such as co-occurrences and semantic similarity between locations).
3. A new network methodology that better incorporate the diversity of actor and relationship types in the data, spatio-temporal information, or for constructing edges from the data and for distributing actor and edge attributes onto the graph.
4. A novel simulation that supports reasoning about patterns in the opiod crisis that uses empirical data to either instantiate the model or to confirm some of the results.
• Generate a new empirical finding that challenges or provides novel support for existing social or political theory, or provides information of policy relevance. Note, the results of computer simulation are viewed as empirical findings.

In addition, a strong entry should be well-written and provide some level of creativity in its use of or combination of data.

What to Submit

You need to submit 5 things - A Challenge Paper, A PDF of your poster, a PowerPoint promotion slide, and for those who have their challenge paper accepted – a full paper and the associated dataset. The challenge paper, poster PDF, PowerPoint promotion slides, and the final paper will go in the on-line proceedings. The winning entries 10 page paper will be published in CMOT. The data will be made public in the disinformation repository.

1. Challenge Paper: A paper describing the project. This should be a maximum of 10 pages including references and figures. This should define:
   • Which of the three question is addressed?
   • How did you focus this question down so that it was manageable?
   • What data was used? How big is it (e.g. number of messages and actors)? What time period does it cover?
   • What is the novel contribution?
   • What is the key methodology or methodologies used?
   • What is the key policy issue or theory being addressed?
   • Who is the team? Provide names, email and affiliations.
   • What tools did you use?
2. A PDF of the poster:

   This will be put online. You are, however, responsible for printing and bringing your own poster to the conference. An easel will be provided, but not posterboard.

   The poster should be either 4’x3’ or 3’x4’.

3. Promotion Slide: A one page promotion slide to be used to entice people to come see your paper. This is a single PowerPoint slide. The purpose of this slide is to excite people to come to your poster. This slide will also be put on line. You will be given one minute to present this slide at the conference to encourage people to come and see your poster. This slide should contain:
   • Title of project
   • Names of all team members
   This slide may contain:
   • Any word or image or idea that you think will promote your poster
   • Logos for your group, company or organization

When to Submit

Challenge Response Submission: 17-May-2019, At this point just send the short abstract.

Author Notification: 01-June-2019

Final Version Challenge Response Submission (10 page paper, PDF of poster, promotion slide, data set): 25-June 2019

Video Presentation: During conference. You should be prepared to give a short video presentation describing your response to the challenge problem. You can use slides with this if you wish.

How to Submit

All challenge participants will need to submit these items:

1. Short Abstract: Due May 17th. This is a minimum of 2 pages and a maximum of 6 pages including references and figures. It should address what was done, how it was done, what data was used, and how this met the challenge.
2. One page slide: This is a synopsis slide that will be used in the 1 minute teaser presentation to get people to come to the poster.
3. PDF of the poster that can be viewed online.
4. Final Challenge paper: Due June 25th. This is a maximum of 10 pages including references and figures. These should not have been submitted elsewhere. These will be put on the conference website as part of the online proceedings which is not archival. In addition, the final paper of the winner, runner up, and potentially other final papers, will be published in a special issue of the journal of Computational and Mathematical Organization Theory which is archival. Submission of a challenge entry constitutes willingness to have the final challenge paper published in the venue.
5. Who is the team? Provide names, email and institution.
6. The abstract, slide, and poster that are student-led need to be clearly marked as student-led. To be considered student-led the following conditions must be met:
   1. The project was led by a student enrolled in a university
   2. The project is not coming out of a corporation, government lab, or FFRDC

The submission website is available at: https://easychair.org/conferences/?conf=sbpbrims2019. Be sure to choose the Challenge track.

What to Present

All entries will send at least one team member to SBP-BRiMS 2018 who will be registered for the conference by the early registration deadline, June 20, to present their poster in the poster-session. The poster will be 4’x3’ or 3’x4’. Participants may bring in additional props to enhance their presentation. In addition, the team spokesperson should be ready to present a 1 minute talk using the slide, to encourage people to come to their poster. Each team will also do a short talk that will be video taped and made available describing their approach and solution. Finally, the winning entry will give a short talk on the last day of the conference.

Download a consent form for videotaping here and submit the form at the registration desk when you arrive for the conference.

How Entries will be Judged

Entries will be judged by community voting at the poster session.

Who is Eligible

Anyone with an interest in using this data to address a social or policy issue. Entries are accepted from single individuals or teams.

Note all SBP-BRiMS program committee members, all challenge committee members are eligible. Why? Because the vote is done by community voting.

The final paper for the winning entry will be published in the journal Computational and Mathematical Organization Theory – in the SBP-BRiMS 2018 special issue. In the case of a tie, both papers will be published.

A member of the team that developed the winning entry will do a short presentation on the final day of the conference describing their response to the challenge problem.

The following data is publicly available and contains some information of relevance to the challenge. Participants may use this or other data.

• Kaggle: https://www.kaggle.com/mrisdal/fake-news/data
• KaiDMML: https://github.com/KaiDMML/FakeNewsNet
• Github: https://github.com/BuzzFeedNews/everything
• LIAR dataset: https://www.cs.ucsb.edu/william/data/liar_dataset.zip
• BS Detector: https://github.com/bs-detector/bs-detector
• Dataworld: https://data.world/datasets/fake-news

• Kathleen M. Carley
• Nitin Agarwal

All questions and concerns can be sent to sbp-brims@andrew.cmu.edu

Kai Shu, Amy Sliva, Suhang Wang, Jiliang Tang, and Huan Lu, “Fake News Detection on Social Media: A Data Mining Perspective,” ACM SIGKDD Explorations Newsletter (2017): arXiv:1708.01967.

Starbird, Kate, Jim Maddock, Mania Orand, Peg Achterman, and Robert M. Mason. "Rumors, false flags, and digital vigilantes: Misinformation on twitter after the 2013 boston marathon bombing." iConference 2014 Proceedings (2014).

Uberti, David. “How Misinformation Goes Viral: A Truthy Story.” Columbia Journalism Review, September 3, 2014.

Aditi Gupta, Hemank Lamba, Ponnurangam Kumaraguru, and Anupam Joshi. Faking sandy: Characterizing and identifying fake images on twitter during hurricane sandy. WWW ’13 Companion, pages 729–736, 2013.

Matthew Benigni, Kenneth Joseph and Kathleen M. Carley, 2017, “Online Extremism and the Communities that Sustain It: Detecting the ISIS Supporting Community on Twitter,” PLOS ONE, 12(12), e0181405

Samer Al-khateeb and Nitin Agarwal. Examining Botnet Behaviors for Propaganda Dissemination: A Case Study of ISIL's Beheading Videos-based Propaganda. In Proceedings of the Behavior Analysis, Modeling, and Steering (BEAMS 2015) co-located with the IEEE International Conference on Data Mining (ICDM 2015), November 14-17, 2015, Atlantic City, New Jersey.