First, the background
Big data industry refers to related economic activities focusing on data production, collection, storage, processing, analysis and service, including data resource construction, development, sales and leasing of big data software and hardware products, and related information technology services. At present, smart medical care, smart cities, precision poverty alleviation and other related high-tech industries are all inseparable from the support of big data, and big data technology has been widely used in China.
(1) The state implements the big data strategy and builds a digital China.
Big data is considered as the "new oil of the future" and compared to the "diamond mine" in the 21st century, which plays an important role in social production, circulation, distribution, consumption activities and economic operation mechanism. In 2014, big data was first written into the government work report; In August 2015, the State Council promulgated the "Action Program for Promoting the Development of Big Data", and big data officially rose to the national development strategy. Subsequently, the state introduced a series of big data policies, covering multi-level downstream application markets such as ecological environment big data, agricultural big data, water conservancy big data, urban big data, medical big data, transportation and tourism service big data, and accelerating the implementation of the national big data strategy.
At the same time, with the introduction of the big data policy, local governments have successively set up big data management institutions to promote the development of the big data industry, and more than 200 cities and provinces across the country have successively set up big data management departments.
Figure 1 Number of big data management institutions in each province (unit: one)
(2) The big data industry is developing rapidly and the industry scale is huge.
In 2016, the Ministry of Industry and Information Technology issued the "Big Data Industry Development Plan (2016-2020)", and the national big data industry construction set off a boom. At present, eight big data comprehensive experimental zones have been formed and more than 100 big data industrial parks have been built. With the development strategies of new generation information technology, smart city and digital China, the digital transformation of social economy has been gradually promoted, the industrial support of big data has been strengthened, the application scope has been accelerated, and the industrial scale has achieved rapid growth.
The survey results of 1572 enterprises show that enterprises pay more attention to data analysis, 65.2% of them have set up data analysis departments, and 24.4% of them are planning to set up related data departments.
Nearly 40% of enterprises have applied big data. Among the enterprises surveyed, there are 623 enterprises that have applied big data, accounting for 39.6%, and the application of big data in vertical industries such as finance is increasing. In addition, 24.3% of enterprises indicated that they will apply big data in the coming year.
The investigation on the selection of data analysis methods shows that 40.3% enterprises take real-time processing of dynamic data and provide analysis results, accounting for the highest proportion; Secondly, it is to analyze historical data and make auxiliary decisions through machine learning, accounting for 32.3% and 25.5% respectively. In the near future, with the development and popularization of artificial intelligence technology, the proportion of enterprises that choose machine learning for decision-making is expected to further increase.
On May 6, 2019, China Information and Communication Research Institute released the White Paper on the Integration and Development of Big Data and Real Economy in China (2019), which integrated various factors such as the domestic and international environment and the development of emerging technologies, and estimated the growth rate of China’s big data industry in 2018 to be about 15%, with an output value of 540.5 billion yuan. According to CCID data, the scale of China’s big data industry in 2018 was 438.45 billion yuan, a year-on-year increase of 23.5%; By 2021, the scale of China’s big data industry will exceed 800 billion yuan.
Figure 2 Scale of China’s big data industry from 2016 to 2021 (unit: 100 million yuan)
From the perspective of corporate business layout, the big data industry is mainly concentrated in North China, East China and Central South China.
Table 1 Regional Distribution of China’s Big Data Industry in 2018
(C) The pace of data resources capitalization has steadily advanced.
In August 2015, the State Council issued the "Action Plan for Promoting the Development of Big Data", which clearly stated that "the open sharing of government data should be accelerated and the integration of resources should be promoted". Through the integration and utilization of data resources, all sectors of society have accelerated the process of data circulation sharing and data resource utilization. In October 2018, the Data Management Capability Maturity Assessment Model was released and implemented, which standardized the data management and application work of various organizations and institutions and improved the domestic data management and application capabilities. In October 2019, at the fourth plenary session of the 19th Central Committee in communist party, China, the Central Committee publicly pointed out for the first time that "the mechanism for labor, capital, land, knowledge, technology, management and data and other production factors to participate in distribution according to their contributions should be improved." This is the first time that the central government has publicly proposed that data can be used as a factor of production to participate in distribution according to contribution, which reflects that with the acceleration of digital transformation of economic activities, the multiplier effect of data on improving production efficiency is prominent and it has become an important change of the new factor of production with the most characteristics of the times.
(D) Technology convergence has become the mainstream of big data development.
At present, big data-related technologies have basically matured and gradually become a supporting infrastructure, and its development direction has also begun to shift to improving efficiency and focusing on personalized upper-level applications. With the implementation of 5G communication standards, the Internet of Things, mobile Internet, big data and traditional industries will be deeply integrated, and the trend of technology integration such as computing power, batch processing, TA, module, cloud number and digital intelligence will become more and more obvious. A large number of talents who know both big data technology and other related industries are playing more and more roles in the field of big data applications.
(E) Data security is widely concerned by the industry.
In recent years, security incidents have been exposed in the big data industry. On September 6, 2019, Hangzhou Magic Scorpion Data Technology Co., Ltd., a big data risk control platform located in Hangzhou, was under the control of the police, and the executives were taken away, and related services were temporarily paralyzed. On the same day, another senior executive of Xinyan Technology Artificial Intelligence Technology Co., Ltd., which provides big data risk control services, was taken away to assist in the investigation. The problem of big data security and compliance, especially the protection of personal information, has become a hot spot in the whole society and industry.
With the global tightening of data compliance policies, China’s data legal supervision has become increasingly strict and standardized. Since 2019, the legislative process of data security has been significantly accelerated. The Central Network Information Office has successively issued drafts for comments on four management measures on data security, including network security review, data security management, children’s personal information network protection and personal information exit security assessment. These data security laws and regulations in China focus on the protection of personal information, and the overall compliance of the big data industry will inevitably take this as the core.
Second, the definition of occupation and work tasks
In recent years, with the economic and social development, scientific and technological progress and industrial restructuring, new industries, new formats and new models have bred many new occupations. With the application of big data technology in all walks of life, our society needs more and more big data engineers and technicians. Its job definition and tasks are as follows:
Professional definition of big data engineers and technicians: engineers and technicians who engage in technical research such as big data collection, cleaning, analysis, governance and mining, and use, manage, maintain and serve them.
Main tasks of big data engineers and technicians:
1 research and development of big data collection, cleaning, storage and management, analysis and mining, presentation and application and other related technologies;
2. Research and apply the architecture, technology and standards of big data platform;
3. Design, develop, integrate and test big data software and hardware systems;
4. Big data collection, cleaning, modeling and analysis;
5. Manage, maintain and ensure the stable operation of big data systems;
6. Monitor, manage and guarantee the security of big data;
7. Provide technical consultation and technical services for big data.
Third, the analysis of the current employment population
This report is based on the analysis of the human resources of 27 typical enterprises in the big data industry in April 2019.
(1) Education level
The academic level of big data talents is divided into four categories, namely, master’s degree and above, undergraduate, junior college and junior college.
Figure 3 Academic Structure of Big Data Talents (Unit: Person)
It can be seen that undergraduate courses account for the highest proportion, followed by master’s degrees and above, and specialist courses account for only 12.22%. The big data industry is an emerging industry, and the current academic requirements are relatively high.
(2) Professional sources
Professional sources are divided into four categories, namely, mathematics and physics, economic management, computer and other majors. Computer science accounts for the highest proportion, followed by mathematics and physics. See the figure below for the number and proportion of professionals in the project team who investigate enterprise big data talents.
Figure 4 Professional Source of Big Data Talents (Unit: Person)
(3) Sources of channels
The channel sources of big data talents are divided into four categories, namely school recruitment, social recruitment, internal training and recommendation, and recruitment of training institutions. See the figure below for the number and proportion of corporate big data talents from all sources.
Figure 5 Source of Big Data Talent Channel (Unit: Person)
Among them, social recruitment accounts for the largest proportion, which is higher than the sum of school recruitment, internal training and internal promotion, and recruitment of training institutions. At present, big data talents mainly rely on social recruitment, indicating that school education is out of touch with social needs, and internal training and training can not meet job requirements.
(D) Distribution of salary level
At present, the salary of big data talents is at a relatively high level. The salary is below 10,000 yuan, accounting for 34.6% of the total number; 10,000-20,000 yuan accounted for 35.64%; More than 20,000 accounts for 29.77%.
Figure 6 Distribution of Big Data Talents’ Salary Level (Unit: Person)
(5) Type and quantity of posts
At present, the big data jobs provided by enterprises can be divided into the following categories according to the requirements of work content:
① Primary analysis, including business data analysts and business data analysts. ② Mining algorithms, including data mining engineers, machine learning engineers, deep learning engineers, algorithm engineer, AI engineers, data scientists, etc. ③ Development and operation classes, including big data development engineers, big data architecture engineers, big data operation and maintenance engineers, data visualization engineers, data acquisition engineers, database administrators, etc. ④ Product operation category, including data operation manager, data product manager, data project manager and big data sales. See the figure below for the number and proportion of the four types of jobs.
Figure 7 Structure of Big Data Position Types (Unit: Person)
Fourth, the industry demand for talents
(A) the overall demand
At present, informatization has a profound impact on human economic activities, and it is infiltrating into all aspects of production and life. Data has become a new factor of production. The big data industry has become an important mode for people to use information processing, information storage and information interaction resources on demand, and it is also an important platform for big data processing and deep mining. The role of big data engineers and technicians will become increasingly prominent in China at this stage and in the future.
Big Data Industry Development Plan (2016-2020)It is pointed out that the construction of big data talent team needs to be strengthened urgently, and there is a shortage of talents in basic research, product development and business application of big data, which is difficult to meet the development needs. It is necessary to build a multi-level talent team and establish a talent training and evaluation mechanism that meets the needs of big data development. Strengthen the training of big data talents, integrate the resources of universities, enterprises and society, promote the establishment of innovative talent training mode, and establish and improve a multi-level and multi-type big data talent training system.
According to Tianfu Big Data International Strategy and Technology Research Institute(referred to as "Tianfu Big Data Research Institute") According to the "2018 Global Big Data Development Analysis Report", in 2018, China’s big data industry talents accounted for 0.23% of the total employed population, about 1.794 million.
According to the Employment Trend Report of AI& Big Data Talents in China in 2019, the gap of big data talents in China is as high as 1.5 million in 2019. According to the statistics of the Data Analysis Committee of China Business Federation, the talent gap of basic data analysis in China will reach 14 million in the future.
Figure 8 Scale and growth rate of big data talents
With big data, Internet of Things,With the continuous development of the application of technologies such as 5G, the demand for employees in this profession is increasing. It is estimated that the demand for talents in China’s big data industry will reach 2.1 million in 2020, and the demand for big data talents will still maintain a growth rate of 30%-40% before 2025, with a total demand of about 20 million people.
(B) the impact of industry development on big data-related jobs
From the perspective of format change, enterprises need a large number of compound talents, that is, talents who can comprehensively master mathematics, statistics, data analysis, machine learning and natural language processing. From the perspective of technological changes, the development of emerging technologies such as deep neural networks has made up for the shortcomings of traditional analysis and mining technologies in the era of big data, which requires talents with big data skills to master the relevant knowledge of deep learning and adapt to the needs of big data analysis and mining. From the perspective of operation mode, the change of operation mode requires operators to improve their preparation before operation, grasp during operation, feedback and correction after operation, and improve their foresight and control ability.
At present, the positions related to big data talents at middle and higher vocational level are mainly: data analyst, mining engineer and deep learning./Algorithm/Machine Learning Engineer, Big Data Development Engineer, Big Data Architecture Engineer, Big Data Operation and Maintenance Engineer, Data Visualization Engineer, Data Acquisition Engineer, Database Administrator, Data Operation Manager, Data Product Manager, Data Project Manager and Big Data Sales Engineer.It can be seen that the development of the industry has triggered a technological revolution, and the corresponding posts and requirements have also changed.
(1) The technical level has gradually changed from "kaleidoscope" to "China characteristics" and "Made in China". China standard has gradually become the industry standard, and China certificate has gradually become the industry certificate.
(2) The in-depth development and horizontal expansion of technology have triggered changes in the demand for talents in enterprises, including the demand for high-tech professionals with re-division of positions and the demand for broad and compound talents at the middle level.
(3) Job responsibilities and skill requirements
According to the survey, the responsibilities of related positions of big data engineers and technicians and the vocational skills requirements for talents with college education or above are as follows.
Table 2 Responsibilities and Job Skills Requirements of Big Data Related Jobs
V. Career development channels
At present, people who have been engaged in database management, mining and programming for a long time, including engineers in traditional quantitative analysts and managers who need to make judgments and decisions through data, can become big data engineers and technicians through certain training or self-study.
Due to the small number of big data talents in China, the data departments of most companies adopt a flat hierarchical model, which is generally divided into data analysts, senior researchers and department directors.Three levels. Larger companies may divide different teams according to the dimensions of application fields, while smaller companies need to hold several positions. Big data engineers and technicians can develop in the research direction and become important data strategic talents of enterprises. In addition, big data engineers and technicians have a deeper understanding of business and products than employees in business departments, and can also turn to product department or marketing department, and even senior management.
VI. Expert Views
Dean of Liangshan Transformation Digital Research Institute and Executive Dean of Hangzhou Digital DreamWorks Research Institute.NiancanhuaData is the most important factor of production in this era. The capitalization, value and service of government, city and industry data resources are the general trend, and the digital transformation of all walks of life is imminent. Training big data engineers and technicians is very sufficient and necessary for implementing the national big data strategy, building a digital China, developing a digital economy, and meeting the massive talent demand of governments at all levels and all walks of life.
Director, Information and Software Services Department, Ministry of Industry and Information TechnologyXie shaofengBig data has opened a new stage of information development, and data has become a key production factor. "Software definition and data drive" play an important supporting role in promoting the transformation and upgrading of manufacturing industry. As the competent department of the industry, the Ministry of Industry and Information Technology has launched a series of measures to promote the development of the big data industry. The next step will be to solidly promote the implementation of the big data strategy, vigorously promote the integration of big data and the real economy, and create new advantages in the international competition of manufacturing industry in the digital economy era.
Academician of China Academy of Engineering and researcher of Institute of Computing, China Academy of Sciences.Ni GuangnanTaking big data as productivity may be better and more comprehensive than taking big data as a kind of wealth. Big data productivity will promote the development of production relations, promote the development of society, create endless wealth, and even cause great changes in the development of our thinking in the future.
Academician of China Academy of SciencesAlax ChenIn the era of computing information, the three landmark technologies are digital computers, integrated circuits and optical fiber communication, and the three highlights of the new generation of information technology are the Internet of Things, cloud computing and big data. Now the trend of big data is vast, and the era of big data has arrived. We must follow the trend of the times, constantly learn new knowledge, keep pace with the times and keep up with the pace of the times! The computer industry will always be the world of young people. For the younger generation, big data is both a challenge and an opportunity, and the infinite scenery is at a dangerous peak!
The main founder of Alibaba Group and the former chairman of the board of directorsJack MaThe emergence of the era of big data has brought mankind into the era of the Internet of Everything, and the ability to reprocess data has far surpassed that of the past, and the understanding of the world will be raised to a new height. Big data has made it possible to predict and plan.
Seven, typical cases
(1) Shao Tianfu: Join in the application of big data and create a wonderful life.
With the government, enterprises and other social organizations more and more closely integrated with the Internet, the self-owned big data computer room can no longer meet the needs of development, and despite the influx of a large number of data companies, it still cannot meet the rapid growth of market demand. Shao Tianfu is keenly aware that the spring of big data is coming.
After graduating from Guangdong University of Technology, majoring in mechatronics, Shao Tianfu plunged into the "blue ocean" of big data management. With his selfless, rigorous and meticulous work spirit and continuous practical exploration, after more than 10 years of hard work, he is now a comprehensive and experienced big data engineer and technician, and has grown into the technical backbone of a data center of a data service company.
(2) Liu Yan:The vanguard of big data applications
Liu Yan, a graduate student, devoted himself to the big family of a public security bureau in Tianjin with infinite love for public security work. After the police, Liu Yan deeply realized the far-reaching influence of big data and scientific and technological information on public security work, and based on his post, he worked hard on innovation and consciously improved his ability to apply big data and information technology. Self-taught theoretical knowledge such as JAVA language, Android programming, database construction, etc., and applied the theoretical knowledge to practice, independently developed the "Eagle Eye" mobile phone APP system, made full use of Internet information resources, and combined with some business data, created a personalized business platform for all employees to share and communicate, which was convenient for the police who went out to handle cases to consult at any time. The graphic data module is added to the APP, which makes it really an important tool for graphic investigation and provides technical support for case detection. As a scientific research innovator, Liu Yan was selected as a member of the "Public Security Think Tank Team", "Informatization" talent pool and "Big Data Analysis and Mining Application" talent pool in this city.
(3) Song Zhenglong: I love my job and devote myself to innovative research and development of railway big data.
Song Zhenglong, graduated from Dalian Railway Institute majoring in computer science, is now the network technical director of a train depot, mainly responsible for the management, maintenance, application and software development of information equipment in the train depot.
In the past 30 years, he started as an ordinary technician and gradually grew into an expert in network data development. He successively developed the software of "marshalling sequence table" and "freight ticketing program" suitable for stations, which greatly improved the work efficiency and the accuracy of statistical work.
In 2014, the unit where he worked put forward the idea of creating a digital management platform and implementing full coverage closed-loop management. Song Zhenglong was appointed to undertake the task of project planning, research and development and construction. In order to design and make a good information management platform, he organized research and development during the day and worked hard to build models at night, and led the development team to independently develop management platforms, early warning and prevention platforms, operation control platforms and electronic tables and books platforms. Finally, the deep integration of civil air defense, physical defense and technical defense was realized, the innovation of enterprise management was promoted, and the innovation and upgrading from "digital car connection" to "intelligent car connection" was accelerated, which was highly recognized by the leaders and brother units of the group company.
(D) Jing Qi: an expert who is committed to promoting the informationization of public security science and technology.
Jing Qi is high flyers from the Computer Science Department of Jilin University. After taking part in public security work, she devoted herself to promoting the information construction of public security science and technology. In 2009, Jing Qi was evaluated as a high-level professional and technical talent in Shenzhen, and was transferred to the provincial public security department and the municipal public security bureau for many times to participate in the research and development and construction of public security application systems.
In order to solve the problems that the information of various business systems at the grass-roots level of public security is not interoperable and repeatedly entered, Jing Qi was appointed as the deputy head of the information collection team of the unit. After repeated discussions with the members, he finally established the general idea of opening up, merging and streamlining various line systems with the transformation of the police integrated information system as the core and the law enforcement case handling process as the main line. In order to complete the task well, Jing Qishe takes care of everyone, and she can always be seen in the front line of research, meeting, communication and coordination. In the end, the team led by Jing Qi realized the goal of "one input and sharing by the whole police", which effectively improved the efficiency of public security work.
When a police station reported that the auction of the items involved was time-consuming and a waste of police force, Jing Qi immediately led the team to discuss the solution overnight. After more than a dozen consultations with the person in charge of the price center, the price identification system was finally docked with the police comprehensive system. Now, the police handling the case can stay at home, just initiate an application through the police comprehensive system and submit the necessary information, and the price determination center can directly feed back the determination results through the system. This work alone can reduce the workload of Shenzhen Police Force by more than 10,000 days every year.