Blackout: Engineering Firm Exposes Critical Infrastructure Data

Updated on August 7, 2017 by Dan O'Sullivan

Filed under: infrastructure, cloudleaks, rsync

The UpGuard Cyber Risk Team has discovered a new data exposure within the systems of Texas-based electrical engineering operator Power Quality Engineering (PQE) , revealing the information of such clients as Dell, the City of Austin, Oracle, and Texas Instruments, among others. Left accessible to the wider internet via a port configured for public access and used for rsync server synchronization, the breach allowed any interested browser to download sensitive electrical infrastructure data compiled in reports by PQE inspectors examining customer facilities.

With a poor CSTAR external cyber risk score of 181 out of a possible 950 at the time the exposure was discovered, PQE presents a number of potentially damaging attack vectors with this exposure. Beyond this highlighting of potential weak points and trouble spots in customer electrical systems, publicly downloadable schematics reveal the specific locations and configurations of government-operated top secret intelligence transmission zones within at least one Dell facility. In addition to this exposed customer data, a plain text file of internal PQE passwords was also stored in the repository, potentially enabling further access to more company systems.

This exposure illustrates several pertinent and common issues driving the spread of cyber risk today. The configuration of PQE’s rsync process to allow public access through an open port is an all too common state of affairs in IT environments. While IT personnel can restrict port access to only authorized PQE employees, such measures can easily be forgotten without processes in place to ensure security gaps are identified and closed immediately.

With growing public awareness of the increasing plausibility of cyber assaults on critical infrastructure, exposed electrical data could be of growing utility to malicious actors seeking to attack corporations and public services. The exposure of sensitive, specific data about top secret data handling facilities within an enterprise IT environment further shows the risks of third-party vendors entrusted with highly prized information. Gartner estimates that three quarters of the world’s top 500 companies will, by 2020, consider such vendor risk to be a board-level concern, for reasons which continue to become apparent.

The Discovery

On July 6th, 2017, UpGuard Director of Cyber Risk Research Chris Vickery discovered an open port configured to accept packets at an IP address which, when entered into a command-line interface, returned a fully downloadable data repository originating from Power Quality Engineering. Containing such folders as “Clients,” “User,” and “Intuit,” the full size of the repository is unknown. In an indication of the exposure’s potential scope, however, Vickery had downloaded a 205 GB portion of data from the repository at the time PQE secured its systems on the evening of July 8th, shortly after being notified by UpGuard.

The exposed port granting public access to these systems, 873, is the default port used for rsync (remote synchronization), a command line utility that allows for the easy and rapid copying of data to another machine. While the IP addresses able to access these systems via this port can be easily restricted by IT administrators using rsync’s “hosts allow/deny” functions, this requires an extra step once the rsync utility is configured. This default accessibility, while simple to restrict, can be missed.

The PQE repository, as such, was fully downloadable to anyone connecting to the uncovered IP address, exposing the data of a number of apparent PQE customers in the process. Within the “Clients” folder in the main repository are folders titled with the names of a number of well-known corporations and public-sector organizations with a presence in Central Texas, such as computer manufacturer Dell, software giant Oracle, telecom carrier SBC, and semiconductor manufacturers Freescale (now owned by NXP) and Texas Instruments, among others.

Image of the main “Clients” folder in the repository.

This data consists of reports and infrared imagery of weaknesses in clients’ power infrastructures as discovered and evaluated by PQE inspectors. Such infrared studies and their associated reporting reveal, with high levels of specificity, energy infrastructure inspection results of clients like HealthSouth Rehabilitation Hospital of Austin.

Image of redacted report, complete with infrared analysis, of electrical system at HealthSouth Rehabilitation Hospital of Austin.

Even more remarkable are the contents of Dell folder 6807, with a document labeled “Director of Central Intelligence Directive No. 6/9” serving as a startling indicator of how sensitive the data entrusted to third-party vendors can be. Emanating from the Director of Central Intelligence—which, until 2005, referred to the director of the Central Intelligence Agency (CIA)—the “Physical Security Standards for Sensitive Compartmented Information Facilities” are detailed at length, for the purposes of installation and configuration in the many far-flung locations in which such rooms are found.

Exposed schematic of the layout of a SCIF in a Dell facility in Central Texas.

What is a Sensitive Compartmented Information Facility, or “SCIF”? A SCIF is a painstakingly-designed secure room used by security-cleared individuals to receive sensitive information. Constructed with the specific goal of making external surveillance, eavesdropping, or interception of any information in the room as difficult as possible, SCIFs are common to intelligence community facilities and military installations. The White House “Situation Room” is, in fact, a SCIF, as are rooms constructed in the Capitol and in Trump Tower for use by intelligence agencies in briefing authorized elected officials.

Per the documents exposed, among the locations in which such a SCIF is located in a Dell facility in central Texas. Schematics reveal the room’s precise location within the building, down to which area of the SCIF is allotted for “Top Secret” communications. The documents confirm the exquisitely stringent standards for the construction of such a room, complying with TEMPEST-level security standards for any acoustical or radio transmissions, and extending to such detailed specifications as the construction of intrusion-defeating air ducts surrounding the SCIF.

Besides these reports, other exposed data for clients, such as that of the City of Austin, include schematics of solar fields, electrical gap analyses, proposals for future construction, inspection reports of aviation breakers at local airfields, maintenance reports for municipal fuel systems, and a “Hazardous Operations Report.” This report contains a detailed risk characterization table and schematics for Austin Energy Sandhill Energy Center.

The cover of the Hazardous Operation Report.

Also stored in many of the “Client” folders are assorted sensitive documents, such as purchase orders, supplier qualification forms, and non-disclosure and confidentiality agreements signed by both client executives and PQE representatives.

Within the repository’s “User” folder, a document titled “computer stuff.docx” lists a number of plaintext PQE passwords, potentially enabling the unauthorized access of these other PQE internal systems.

Redacted screenshot of “computer stuff.docx” and the sensitive access data contained therein.

The indication that at least one password is for PQE’s GoDaddy webhosting account raises the frightening possibility that the firm’s website could have been accessed and exploited, perhaps funneling visitors into a watering hole attack. If client data was stored on any of these networks, these clients could also have been further exploited.

The Significance

The PQE data exposure presents a uniquely varied illustration of the many attack vectors a malicious actor can take in 2017 to exploit the sensitive data of enterprises for their own purposes. Of prime importance, however, is the process error which resulted in the data being exposed in the first place: the configuration of the rsync port to be open to public access.

Enterprises must maintain processes to ensure that system permission is only granted to those users who should have access. Rsync directives such as auth users, strict modes, and allow/deny users, as well as tools like firewall ACLs, can significantly and effectively reduce the attack surface available to malicious actors. In short, while the effects of such an exposure can be hugely damaging, the precautions to prevent it are relatively simple, free, and already available.

Indeed, the cyber risks presented by such an unsecured state are numerous. As already indicated, the potential exposure of keys to internal PQE systems could allow hackers to access whatever other data has been entrusted to the firm. PQE’s extremely poor 181 CSTAR score indicates a great deal of risk around the systems employed by the company, though that number’s rise to 428 in the wake of being alerted by UpGuard is a positive development.

Cascading breaches, in which an initial exposure allows successive penetrations of internal IT systems, is a real threat, spreading the risk taken on by any one enterprise to any other enterprise that has entrusted the affected systems with its data. This is the essence of third-party vendor risk: if you are giving your privileged data to a third party, you are exposing yourself to whatever cyber peril that third party has put itself in, as if you had done so yourself. The actual CSTAR scores of the websites of entities affected by this exposure vary, but with an overall trend towards the low scores of poor security configurations:

Dell (CSTAR 428)

The City of Austin (CSTAR 547)

Oracle (CSTAR 866)

Texas Instruments (CSTAR 576)

Apart from Oracle, each of these affected entities have low to mediocre CSTAR scores. But it need not be an enterprise’s own systems that expose sensitive data; PQE, with its score of 181, illustrates the real risk of handing data over to organizations with clear signs of serious cyber risk.

In the case of PQE, the consequences could have been severe. The exposure of the location and configuration of a SCIF could have provided malicious actors with a target for stealing classified information. In addition, there exists stark evidence of the growing danger of cyber attacks that cripple medical facilities or power grids. This state of affairs, while perhaps sounding fantastical, is real, and carries the threat of endangering people’s lives. With the stakes higher than ever before, enterprises must ensure that they are doing all that they can to build processes which value and protect the integrity of their data.