UpGuard can now disclose that a storage device containing 1.7 terabytes of information detailing telecommunications installations throughout the Russian Federation has been secured, preventing any future malicious use. This data includes schematics, administrative credentials, email archives, and other materials relating to telecom infrastructure projects.
Until recently the files were hosted on a rsync server configured for public accessibility. While documents and data stemming from several major Russian telecommunications providers are present, the primary entities affected by the exposure appear to be Nokia and Mobile TeleSystems.
In an email to UpGuard, Nokia states the data set “was a hand-over folder” from a Nokia employee to an unnamed third party. The unnamed third party then “failed to follow his company’s business processes, security policies and his personal responsibility to protect it.” The rsync server was not directly hosted by Nokia.
In addition to the risk posed by any large scale exposure related to telecommunications, the data set also includes photographs and installation instructions for SORM (System for Operative Investigative Activities), the hardware which enables communication interception and review by Russian law enforcement agencies such as the FSB.
Table of Contents
- MTS: Russia's Largest Telco
- Data Overview
- Potential for Misuse
- Notification and Response
- Executive Summary
Mobile TeleSystems–abbreviated as MTS in English and MTC in Russian– may not be a familiar name outside of Eastern Europe, but in their region they are the preeminent telecom operator. MTS has the largest telecom market share in Russia and over 100 million subscribers to their mobile network, the most of any company in Russia. In their most recent financial reporting, they note that “MTS was recognized as Russia’s most valuable TMT [technology, media, and telecom] brand and the country’s ninth most valuable brand overall.”
Also of note in their Financial and Operating Results for Q2 2019 is an item explaining their planned 90 billion ruble CAPEX spending. Part of that sum will go toward satisfying the data storage requirements of the “Yarovaya Law,” which directs telecom operators to store voice and SMS messages for up to six months. While the data exposed here does not pertain to those future plans, a portion of the data set does concern an even larger telecom infrastructure project: the installation of hardware for the “System for Operative Investigative Activities,” known by its acronym SORM.
SORM is the system by which telecommunications can be intercepted and inspected by the FSB and other law enforcement agencies. Russian authorities utilize this special gateway to monitor, log, and enforce blacklist censorship on traffic passing through the service provider's network. User IDs, emails, text messages, IP addresses, and phone numbers are among the details accessible to the SORM system. Since 1995, telecom providers have been required to install SORM hardware devices, and as technology has advanced, so has the specification for SORM. In 2014, a new generation of equipment known as SORM-3 was mandated, and companies like MTS had to comply, requiring a nation-wide infrastructure refresh.
Much of the data exposed in this collection details the 2014-2016 installation of SORM hardware by Nokia Siemens Networks, in coordination with MTS. A project of this size could not be carried out alone. Dozens of other companies were also involved– one spreadsheet titled “AllProjects.xlsx” lists 64 subcontractors– but our review of the contracts and communication documents indicates that Nokia provided high level technical expertise and implementation proposals. At the time, Nokia had recently come under criticism for their contributions to state surveillance in Bahrain and Iran, including cases where dissidents were known to be imprisoned and tortured. While the lawsuit against Nokia was dropped and Nokia withdrew from taking new projects in Iran, they have a proven track record for installing so-called “lawful intercept” systems.
Exposing any data related to a system with the power and secrecy of SORM to the public internet is an event; leaking what appears to be an inventory of the most recent generation of installed hardware for a nation’s largest telecom provider is unprecedented. To give one indication of the level of security expected for SORM equipment, “providers are required to pay for the SORM equipment and its installation, but they are denied access to the surveillance boxes.” Not even MTS is allowed access to SORM boxes installed within their own facilities, but anyone with an internet connection could have downloaded the exposed documents revealing system architecture, installation sites, and credentials.
One way to begin comprehending the scale of this exposure is with a listing of the number and storage size of the largest file types. What makes data sensitive is the qualitative nature of the information stored inside it, but even purely quantitative measures can sketch out the contours of the exposure’s impact. Examples of some of the most prevalent document types are included to illustrate the significance of exposing these quantities of data.
Of the 1.7 terabyte total, 700 GBs were photographs stored as JPG images. These 578,000 photos show a vast inventory of Russian infrastructure hardware. Rare views range from inside data centers, to the tops of antennae hundreds of feet tall, to high resolution images taken close enough to show legible barcodes, serial numbers, and locale-specific engineering documentation. The photographs highlight how physical and information security are increasingly intertwined. Information that would normally require penetrating several layers of physical security can be gathered from thousands of miles away when those records are digitized and stored insecurely.
Close up image of a SORM box without identifying contextual information.
The data set includes email archives totalling 245 gigabytes. Outlook data files in PST format are archive backups or offline stores of Microsoft Outlook email, calendaring and contact data. Such communications can include logistical planning, sensitive attachments, private conversations, and even plain text credentials. While nothing extraordinarily noteworthy has stood out among the emails, the language barrier offers more than the usual number of challenges related to processing and reviewing these particular email archives.
The vast majority of the 197,343 PDFs appear to contain contractual agreements between telcos and the companies contracted to install and maintain physical hardware. Many of these agreements are accompanied by approval signatures from government body officials.
Because these archives are compressed, the true file size of the dataset is even larger than it appears. Backups of document stores, project proposals, operating manuals, progress reports and at least one desktop archive are present. The archive files we reviewed included bootloaders and other software for use with the associated hardware.
Schematics and designs of network equipment such as industrial sized antennae and floorplans. These include system information, engineering documents, and sensitive location details for the many types of network devices involved in the exposed infrastructure.
Common office spreadsheets used to view and manipulate data. These contain inventories of network gear, such as switches and routers, with information like IP addresses, names of employees assigned to install equipment, progress notes, and tips regarding how to physically enter project sites.
The DOC files present generally contain draft copies of contract and proposal documents. In DOC form the files are not signed. Signed copies are present in PDF format.
BAKs are backup filetype usually affiliated with databases such as Microsoft SQL Server. In this case the BAKs we reviewed were backups of schematics otherwise stored in the DWG format.
Unlike the PST files, which are larger archived collections of email, MSG files are individual emails that have been saved as text. Each MSG file typically represents a single piece of correspondence, or perhaps a thread of messages if they were included at the time of sending. The MSG files found within this repository appear to be project-specific notes a coworker would find useful to reference. This purpose would coincide with Nokia’s statements regarding how the files came to be exposed.
Access is a common database application used to store various types of data. These Access databases contain details such as employee and subcontractor names, phone numbers and other contact information, as well as macros which would normally query more intricate database details from other servers. Their full purpose is not known however, due to UpGuard not typically running arbitrary macros found while analyzing data sets.
Taken as a whole, this collection of data represents the kinds of artifacts generated by large scale hardware and software deployment projects, from the business agreements that authorize the project, email communications used to discuss and plan activity, schematics and architecture documents for installation, photographs collected while surveying and auditing project sites, and the archival files storing artifacts for future reference, maintenance, or restoration purposes.
At an abstract level, this collection of files is somewhat similar to other enterprise-scale data exposures such as:
- Data Warehouse: How a Vendor for Half the Fortune 100 Exposed a Terabyte of Backups
- Public Domain: How Configuration Information For the World's Largest Domain Name Registrar Was Exposed Online
- Short Circuit: How a Robotics Vendor Exposed Confidential Data for Major Manufacturing Companies
In this case, though, the contents of the exposed files pertained to the inner workings of one of the world’s most advanced state surveillance systems.
As mentioned, SORM involves the installation of hardware running specialized software, and the presence of relevant details in this repository decreases the security of both layers. The SORM installations in scope for this collection of projects pertain to at least sixteen cities: Vladimir, Lipetsk, Ivanovo, Kaluga, Kostroma, Bryansk, Smolensk, Ryazan, Belgorod, Voronezh, Kursk, Oryol, Tula, Tver, Tambov, and Yaroslavl, in addition to Moscow. The schematics and documentation include information detailing the power distribution units and batteries which run the systems. If ambitious adversaries were to seek ways in which to go from digital compromise to physical facility harm, these are the types of documents that would provide an initial roadmap toward that goal.
Directory named "Audit" with thousands of subfolders for installation sites.
In what appears to be a means for centralizing information, the SORM system documents illustrate a network layer that makes the data accessible to law enforcement. SORM system documents show the hardware communicating on private subnets only accessible via VPN or other method of privileged access. As with other information technology projects, convenience of use– being able to access data remotely rather than air gapping every appliance to lock its data inside– introduces the possibility of compromising the non-physical security layers.
Heavily redacted file containing a URL and username and password.
In this case, credentials related to administrative platforms were present, raising the possibility of outside access without the necessity of physical compromise. Their exposure introduces the logistical challenge of large scale updates, potentially creating further problems.
Redacted architecture diagram.
After confirming the contents of the server were most likely legitimate, UpGuard began notification efforts to secure the exposed data. UpGuard's first attempt at emailing Nokia took place in the afternoon of September 9, 2019 (to which no response was received). A phone call later that day resulted in a Nokia representative providing a switchboard number which would be active the following morning.
During the morning of September 10th, UpGuard's Director of Cyber Risk Research, Chris Vickery, reached an individual identifying himself as a Nokia Security Manager via the previously provided switchboard number. The Security Manager then informed Vickery that the security manager had "no time to deal with" the data breach notification and should contact the company via their website.
UpGuard later learned the security manager Vickery was transferred to is a physical security manager rather than being of the digital type Vickery assumed he was speaking with.
On September 11, 2019, UpGuard reached out to a U.S. government regulator in order to seek the contact information of someone more receptive at Nokia. The contacted individual was able to facilitate a conversation between UpGuard's Risk Research Team and Nokia's New York law firm attorneys. At 11:20 pm PDT the same day, Nokia's Head of Information Security in Finland called Chris Vickery, who then provided the IP address of the exposed rsync server. The rsync server was still open well into the night of September 12th. When checked again on the morning of September 13th, the files were no longer publicly accessible.
Even as data exposures are endemic to digital business, this case stands out for its potential nation-level consequences. In particular, it highlights the concerns that arise when data exposures intersect with federal systems: whenever power is centralized in software, the inevitable exposure of that information gives whatever power the owner had to unknown third parties. In this case, the SORM system allows Russian investigators granular access to digital messages traversing Russian territory, but the existence of the system also implies consequences of its compromise. Following the exposure of this data to the public internet, these are concerns to be contended with.
Russia is not alone in attempting to surveil communications traveling within its territory, nor in having efforts impacted by data exposures. UpGuard has previously reported on leaks of data from U.S. agencies, including access credentials exposed by Booz Allen Hamilton, a virtual hard drive pertaining to the "Red Disk" project intended to centralize battlefield coordination, and a Department of Defense project found collecting millions of social media posts. The problem of data leaks is not unique to any country or industry; it is an inescapable part of humans operating information technology. The consequences of those data exposures, however, do vary, and the more concentrated and sensitive information becomes, the higher the stakes.
To learn more, continue reading at TechCrunch.
We'll email you a summary PDF containing five key takeaways from this breach and the infographic above.