Beware the Ides of March? Troubling Early Hints of Russian Interference in the U.S. Presidential Election
[UPDATE -- 01/03/17 -- After analyzing the Grizzly Steppe JAR report released jointly by DHS and FBI on 12/29/16 we can confirm that at least 7 IP addresses from their report exactly match IP addresses that were probing the City of Ashland website back in March. Of these seven IP addresses three were from TOR exit nodes, which is an encrypted and anonymous browsing service that is essentially untraceable. We are in the process of submitting more details to journalists and we will publish our own report shortly. UPDATE 2 -- 01/11/17 -- Our subsequent Grizzly Steppe article was published on Jan. 11th, 2017.]
After spotting some odd Russian and Kyrgyzstan-based traffic to the website for the City of Ashland, Wisconsin, we were recently interviewed by a local newspaper about the significance of this strange traffic (ref-01). As we stated in the interview, recent talk about Russia’s potential interference in our presidential elections prompted us to take a more careful look at the various sources of foreign traffic coming to the City of Ashland’s website.
What we found was entirely unexpected. Traffic from both Russia and Kyrgyzstan showed sharp, unprecedented increases to the City website starting around March 16 of this year (figure 1). This traffic remained elevated and sustained through the election. We could not find any other increase like this in the previous two years of log files for the city. When we looked at other countries that are known sources of hacker traffic, like Brazil, India, and Pakistan to name a few, none showed anything other than flat baseline levels of traffic, with no dramatic spikes in March or other months during the presidential election. As we told the Ashland Daily Press, “On most websites you are always going to get a little bit of traffic there, and every day there is always somebody looking for a security issue, so you are always going to see a baseline of traffic that is always a little suspect. Most of the time they don’t find anything, they are just trolling for security flaws.”
The Russian and Kyrgyzstan traffic patterns were different and unusual in that they both started around the same day, March 16. The anomaly was significant enough for us to contact Ashland Mayor Deborah Lewis and to ask if anyone else in the area had seen something similar. Mayor Lewis spoke with Paul Houck, Bayfield County Director of Information Technology, who confirmed that the county had discovered the same pattern of web traffic to its own website. Houck agreed with our initial speculation that the traffic might be related to the U.S. presidential election. Once Houck had confirmed seeing the same thing in Bayfield County, Ashland Mayor Lewis passed this information on to local law enforcement, who in turn passed it on to the FBI.
We also expanded our analysis to look at other non-city and non-county websites in the area and found only one other website that had the exact same pattern: the Madeline Island Chamber of Commerce website. We believe it is possible that this website was mistaken for a municipal or county website. We have also reached out to the city administrator of La Pointe, Wisconsin to see if they observed a similar pattern and they are currently looking into the issue.
As reporter Rick Olvio stated in the original Ashland Daily Press article, “Still it’s difficult to ascribe a non-malevolent motivation for this kind of behavior. It is difficult to see what legitimate interest computer operators in Russia would have in repeatedly hitting on computers in the Chequamegon Bay region” (ref-01). We agree with his assessment.
So why did all of this begin happening in mid-March and what was the motivation or purpose of this traffic? The pattern was certainly unnerving to us, considering that the U.S. Government has now formally accused Russia of hacking the computer networks of both major political parties’ national committees along with multiple party officials and asserting that Moscow was trying to interfere with the U.S. Presidential election (ref-02). Some of our initial analysis of the server logs files revealed that much of the suspect traffic was indeed scanning for known security flaws in common website-related code. No obvious flaws existed on this site, but clearly at least a portion of the Russian traffic was looking for vulnerable code that could be exploited. Some of the other traffic appeared to be information gathering, possibly looking for key people involved with the election apparatus.
We have so far found no evidence of a breach but at the same time aren’t confident we would even be able to identify a state-sponsored hacker. Russia is known to have some of the best hackers in the world. If they can breach a U.S. Department of Defense website, we certainly are not naïve enough to think that we could stop them.
With the Wisconsin presidential recount now ending, the areas related to the odd website traffic in question have all finished their recount. At least in Ashland, Bayfield, Douglas and Iron counties, the recount was done by hand and did not reveal any significant differences in the vote totals or widespread vote tampering. Of course, many counties in Wisconsin did not do a full hand-recount of the paper ballots and instead fed the ballots back through the same machines to be counted again. It’s not yet clear if other Wisconsin counties experienced similar and unusual levels of website traffic coming from Russia or Eastern Bloc countries and starting in mid-March.
Besides providing cyber-security services for small businesses all over the country, our bread and butter service is search engine optimization (SEO). In this case, we reasoned, SEO tools might at least help put the unusual mid-March traffic surge in context to events related to the election, and we used a combination of Google searches and Google Trends to analyze events occurring around the same time. March was particularly newsworthy for many of the presidential contenders and the heart of the primary season (March 15, in fact, was one of the Super Tuesday primaries that included Ohio, Florida and North Carolina).
We started our analysis with a simple Google search that included the names of the top four presidential candidates (Donald Trump, Ted Cruz, Hillary Clinton, Bernie Sanders) and a separate set using their campaign managers at the time. Trump had two different campaign managers in March, Corey Lewandowski and Paul Manafort, so we included both in our analysis.
The first set of searches included the following search criteria: [“full name of candidate or manager” + "march" + "russia"] and we instructed Google to only search within the past year to increase the relevancy of the results. By using the “+” sign and putting the terms in quotes, we also forced the search engine to include only those results with all three terms. Trump’s name appeared most often in our results, shown below (figure 2). This wasn’t entirely unexpected, given his previous statements appearing to praise Russian Federation President Vladimir Putin and his other mentions of Putin on the campaign trail (ref-03).
We repeated the process using a more selective search to further reduce the number of results and weed out peripheral and unrelated results. This search included the following criteria: [“full name of candidate or manager” + "march" + "ties to russia"] and again was limited to the past year. Our results are shown below (figure 3).
This time, we saw that the search results included Trump’s campaign manager, Manafort, four times as often as any other campaign manager. This was certainly an intriguing result. A few Google searches of Paul Manafort and whether he had any potential ties to Russia, revealed numerous in-depth articles like this one from the Atlantic (ref-04). He appeared to have numerous ties to Russian oligarchs, Vladimir Putin and pro-Russian politicians in the Ukraine.
Google Trends is a particularly useful tool that can look for historical spikes in certain topics or searches and zero in on when significant changes occur. We examined the March 1, 2016 to April 30, 2016 time period and specifically looked for spikes in “Paul Manafort” in referenced articles and Google searches. Interestingly, one hit occurred on March 15 and the first major spike then began on March 29, when Manafort was formally announced as Trump’s new campaign manager. Lewandowski was removed as Trump’s campaign manager on March 17 because of an apparent rift with Manafort, who at the time was Trump’s chief strategist (ref-05).
We note that according to CNN, unnamed officials now believe that the initial hack of John Podesta’s email account, the campaign chair for Hillary Clinton, occurred on March 19 (ref-06). This was four days after the unusual traffic we discovered in two small, rural, northern Wisconsin counties. This illustrates a very important point when it comes to hacking. Most hackers that are good find an initial doorway (breach) into the site or network, create some extra backdoors in case the initial breach gets patched and then they wait for months and months to either collect surveillance, carefully steal data from their target or find the perfect time to further exploit their hack. The hack of John Podesta's email is a perfect example of this. The first public revelation of this hack through a wikileaks dump was at the beginning of October, about six months after the initial breach in March. We have repeatedly seen this same pattern in the small business websites that we have helped remediate and harden. Almost always the initial breach was successfully done months prior to the final attack or end goal of the hackers.
Although intelligence officials have gathered evidence of Russian-based hacks of other government and political party-affiliated websites dating back months or even years, March 2016 appears to have been a particularly active period for Russian involvement and/or interference in the U.S. Presidential election. The how and for what purpose at this point is not clear. What is clear however, is that a larger and more extensive investigation needs to be done and other city and county websites across the country need to verify whether they were scanned and/or breached prior to the elections.