Public opinion monitoring system

overview

With the popularization of the Internet, China has more than 338 million Internet users, becoming the world's largest country of Internet users. The importance of the Internet is highlighted, and hot issues of Internet public opinion are emerging in endlessly. The New Deal of registered residence in Shanghai, free medical care for all people in Shenmu County, Shaanxi... are all spread through the Internet at the first time. The Internet has become a direct channel for the government to understand the people's situation, and also an important public opinion position for the government under the new situation. Therefore, once online public opinion is wrongly controlled and guided, it will become a major hidden danger that affects social stability. How to respond to online public opinion is currently testing governments at all levels in China.

Internet public opinion is complicated, and the Internet speech space is2010The year continues to expand, especially“Weibo”The rise of online issues has been responded to and deeply explored by traditional media, and traditional media reports are often quickly fermented and amplified into national public opinion online. Some netizens have begun to intervene in reality through some gentle offline actions. The government has accelerated its response to online public opinion and listed online reporting as a new channel for party discipline, political discipline, and judicial supervision, holding officials responsible for stirring up public grievances accountable. Meanwhile, due to the increased confrontational nature of some emergencies, some places tend to prioritize restricting the flow of online information as a way to control the situation, while other local governments boldly explore new mechanisms for responding to public opinion and handling emergencies

The collection and integration of multiple data sources. At present, news, forums, blogs, microblogs, video websites and other public opinion information dissemination channels can spread the same public opinion information in different dissemination channels, resulting in inconsistent processing of data from different sources by the data monitoring system and inconsistent standards. In order to ensure the efficiency of supervision, it is necessary to standardize and integrate the metadata of the supervision system to form a unified basic database of Internet public opinion information

Multi dimensional monitoring issues, public opinion monitoring requires multidimensional analysis of multiple data sources, and analysis and presentation of different dimensions according to the needs of different users.

The problem of analyzing the correlation and fusion of results is that the existing monitoring systems have relatively little implementation of the analysis function of public opinion information dissemination, only simple analysis of public opinion information text content. Therefore, there is currently no corresponding system and technology for comprehensive analysis of various dissemination channels, news, forums, blogs, microblogs, and related information, resulting in insufficient ability to analyze correlation and insufficient ability to discover potential hotspots. Effective solutions are needed

Massive data processing problem, hundreds of millions of Internet data are generated every dayTBThe above scale of data, how to store massive amounts of data, how to process massive amounts of data in a timely and effective manner, and how to make the system architecture automatically expand with the increase of data scale are all challenging issues faced by massive data processing.

System Characteristics

One is the flexibility of collection. As a globally leading web analysis tool, Web Explorer can accurately locate and analyze the source code structure, node structure, and CSS characteristics of any part of a web page based on its visual features; Flexible handling of any complex page layout, with dozens of built-in data processing methods, easily supporting the extraction of structured data from any semi-structured data;

The second is the timeliness of data collection. Distributed search, meta search, and vertical search complement each other. Search engines adopt distributed and parallel load balancing technology. When the server experiences an imbalance in running load, the system can automatically adjust the load of the server and network devices according to a pre-set adjustment mechanism, fully utilizing existing resources, ensuring stable system operation, improving operational efficiency, and achieving our goal of rapid search data collection;

The third is customizability, which means the ability to focus on monitoring key objects and topics according to users' customized needs;

Fourth, sustainability, that is, tracking the number of subsequent reports, reprints and comments of various Internet media on known topics, and mastering their development scope, trends and trends;

The fifth aspect is comprehensiveness, which involves integrating, analyzing, and processing system data established for different technological forms to ensure the comprehensiveness of data within the jurisdiction.