Skip to main content

PhantomJS / POODLE

On Tuesday October 14th, 2014, Google announced the POODLE vulnerability for SSLv3. Engineering teams scrambled to remove support for SSLv3 on their websites, including our own system operations team.

My team's data engineering system scrapes information from retail banking sites, which we have learned were heavily impacted by the SSL fix on Tuesday. It makes sense that banks would want to keep up-to-date with the latest security issues.

When we scraped pages on Wednesday morning, a large portion of the sites we collect from came up blank in the PhantomJS browser, but without any obvious errors in our log files or in our log analysis dashboard (Kibana).

We were puzzled at this behavior, and when we went to look at the URL's that failed to produce images, it was clear that they were all SSL-enabled and responding with a recent TLS protocol. It turns out that PhantomJS by default only uses the SSLv3 protocol, and so it does not support any of the TLS protocols without additional configuration.

The solution to our problem was to set the command line flag for preferred SSL protocol to any so PhantomJS is free to pick whatever protocol is supported by the site being scraped. While it is unintuitive for PhantomJS to select the least secure protocol as the default, at least we can override it easily.
Labels:

Comments

Post a Comment

Popular posts from this blog

ReactJS, NPM and Maven

I'm just starting to get into working with ReactJS, Facebook's open source rendering framework. My project uses SpringBoot for annotation-driven dependency injection and MVC. I thought it would be great if I could use a bit of ReactJS to enhance the application. If you're looking for a basic conceptual intro, I recommend ReactJS for Stupid People and of course the official documentation  is quite good. In full disclosure, I still have no idea how to do "flux" yet. As an experienced Java backend developer, I'm pretty decent at hacking Maven builds - which is precisely what this blog post is going to be about. First, a word about how React likes to be built. Like many front-end tools, there is a toolkit for the node package manager (NPM). From the command prompt, one might run npm install -g react-tools  which installs the jsx command. The  jsx  command provides the ability to transform JSX syntax into ordinary JavaScript, which is precisely what I want...
21 comments
Read more

IntelliJ Annotations and Maven

IntelliJ has a code inspection feature that is designed to prevent null pointer exceptions based on static code analysis - actually a kind of interesting idea, and the folks over at IntelliJ have recommended that the annotations be included in the Java SDK in the future. I noticed this feature when I was cleaning up some code today and found a cryptic error message in the IntelliJ code inspection tool: Not annotated method overrides method annotated with @NotNull The community documentation for IntelliJ has a nice explanation of this feature , and there is a Maven repository available. The dependency for version 12 of IntelliJ is the following: <!-- STATIC CODE INSPECTIONS -->  <dependency> <groupId>com.intellij</groupId> <artifactId>annotations</artifactId> <version>12.0</version> </dependency> Adding the @NotNull annotation to my overridden method and to a parameter seems to have cleared up the issue. I'm ...
Post a Comment
Read more

Cryptic Facebook Message

Facebook OAuth2 is a feature I frequently integrate, but sometimes its error messages can be downright opaque. In particular I keep on forgetting to associate my Facebook account on applications where Sandbox Mode is enabled. The error message in this case is the following: Sorry, this feature isn't available right now: An error occurred while processing this request. Please try again later. This message is super cryptic, since what I usually need to do is to register my account as a tester or admin on the Facebook application page. If you get this error message here are the steps to fix it: As an application administrator, go to the OAuth2 configuration page for the application in Facebook. Add the Facebook account as an admin or tester on the "edit roles" screen. This will send an application request to that account. As the added account, accept the application request to become an admin or tester. At this point the Facebook login should work for the accou...
1 comment
Read more