Tim Mattison

Hardcore tech

Image Metadata Extraction With AWS Lambda and Java

| Comments

I just started a repo to keep track of my AWS Lambda (Java) experiments. The first experiment is using Java to extract metadata from an uploaded image. It is a bit unconventional because it doesn’t use S3 but that was done to make the setup simpler. Here’s how you can play around with the metadata extractor…

This tutorial will walk you through getting the metadata extractor set up, calling it, and then tearing it down. The commands in this tutorial assume you are in the metadata-extractor directory.

NOTE: Do not forget to do step 5 – “Tear down your function” – or you may be sorry!

Step 1: Build the JAR

1
$ mvn clean compile test package

You should get a BUILD SUCCESS if everything worked.

Step 2: Create the Lambda function

Navigate to step 2 of the AWS Lambda creation process (step 1 is selecting a blueprint/template which we do not want to do).

  • Name your function metadata-extractor
  • Enter a description if you want
  • Set the runtime to Java 8
  • Select Upload a ZIP file (it should be selected by default)
  • Click “Upload” and select the file metadata-extractor-1.0-SNAPSHOT-jar-with-dependencies.jar from the metadata-extractor/target directory
  • Set the handler to com.timmattison.ImageHandler::handleRequest
  • Set the role to Basic execution role or lambda_basic_execution, whichever exists. NOTE: If you have popups disabled this may continually tell you that you didn’t fill the field in. You MUST enable popups to get this to work.
  • Set the memory to 128 MB
  • Set the timeout to 5 seconds
  • Click Next
  • Click Create function

Step 3: Create a public endpoint that you can test

You should now be on a screen that says “Lambda > Functions > metadata-extractor”. There should be five tabs visible. They are “Code”, “Configuration”, “Event sources”, “API endpoints”, “Monitoring”.

  • Click on the API endpoints tab
  • Click Add API endpoint
  • Set API name to LambdaMicroservice
  • Set Resource name to metadata-extractor
  • Set Method to POST
  • Set Deployment stage to prod
  • Set Security to Open (we will destroy this service later so it can’t be abused)
  • Click Add API endpoint

You should now see a table with three columns. The rightmost column “API endpoint URL” contains the URL for the endpoint you just created. It should be similar to this:

1
https://xxxxxxxxxxx.execute-api.us-east-1.amazonaws.com/prod/metadata-extractor

Copy that somewhere where you can get to it later.

Step 4: Upload an image to the URL

Now you’re ready to upload an image to the URL you just created. Lambda is expecting data in JSON format, not binary uploads. We do something a bit hacky to make our lives easier here. We take the image, convert it to base64, and put it into a JSON object before we upload it. This is done with the Python script called convert-image-to-json.py. Assuming your image is called myimage.jpg and your URL is https://xxxxxxxxxxx.execute-api.us-east-1.amazonaws.com/prod/metadata-extractor you would do this to upload your image.

1
$ ./convert-image-to-json.py myimage.jpg | curl -X POST https://xxxxxxxxxxx.execute-api.us-east-1.amazonaws.com/prod/metadata-extractor -d@-

If your image contains any metadata you’ll get a response that contains a bunch of interesting info. Here are the results from one PNG I had lying around:

“[PNG-IHDR] Image Width – 332, [PNG-IHDR] Image Height – 48, [PNG-IHDR] Bits Per Sample – 8, [PNG-IHDR] Color Type – True Color with Alpha, [PNG-IHDR] Compression Type – Deflate, [PNG-IHDR] Filter Method – Adaptive, [PNG-IHDR] Interlace Method – No Interlace, [PNG-iCCP] ICC Profile Name – ICC Profile, [ICC Profile] Profile Size – 1960, [ICC Profile] CMM Type – appl, [ICC Profile] Version – 2.2.0, [ICC Profile] Class – Display Device, [ICC Profile] Color space – RGB , [ICC Profile] Profile Connection Space – XYZ , [ICC Profile] Profile Date/Time – Wed Mar 25 11:26:11 UTC 2009, [ICC Profile] Signature – acsp, [ICC Profile] Primary Platform – Apple Computer, Inc., [ICC Profile] Device manufacturer – appl, [ICC Profile] XYZ values – 0.9642029 1.0 0.8249054, [ICC Profile] Tag Count – 11, [ICC Profile] Profile Description – Generic RGB Profile, [ICC Profile] Apple Multi-language Profile Name – 30 skSK(Všeobecný RGB profil) hrHR(Generički RGB profil) caES(Perfil RGB genèric) ptBR(Perfil RGB Genérico) ukUA(Загальний профайл RGB) frFU(Profil générique RVB) zhTW(通用 RGB 色彩描述) itIT(Profilo RGB generico) nbNO(Generisk RGB-profil) koKR(일반 RGB 프로파일) csCZ(Obecný RGB profil) heIL(פרופיל RGB כללי) deDE(Allgemeines RGB-Profil) huHU(Általános RGB profil) svSE(Generisk RGB-profil) zhCN(普通 RGB 描述文件) jaJP(一般 RGB プロファイル) roRO(Profil RGB generic) elGR(Γενικό προφίλ RGB) ptPO(Perfil RGB genérico) nlNL(Algemeen RGB-profiel) esES(Perfil RGB genérico) thTH(โปรไฟล์ RGB ทั่วไป) trTR(Genel RGB Profili) fiFI(Yleinen RGB-profiili) plPL(Uniwersalny profil RGB) ruRU(Общий профиль RGB) arEG(ملف تعريف RGB العام) enUS(Generic RGB Profile) daDK(Generel RGB-beskrivelse), [ICC Profile] Copyright – Copyright 2007 Apple Inc., all rights reserved., [ICC Profile] Media White Point – (0.95047, 1.0, 1.0890961), [ICC Profile] Red Colorant – (0.45429993, 0.24191284, 0.014892578), [ICC Profile] Green Colorant – (0.35334778, 0.67362976, 0.09063721), [ICC Profile] Blue Colorant – (0.15664673, 0.0844574, 0.719574), [ICC Profile] Red TRC – 0.0070344, [ICC Profile] Chromatic Adaptation – sf32(0x73663332): 44 bytes, [ICC Profile] Blue TRC – 0.0070344, [ICC Profile] Green TRC – 0.0070344” “`

Step 5: Tear down your function

The function you’ve created is now publicly accessible. That means that if someone finds it they can hit it and start costing you real money. It is in your best interest to shut this function down when you’re done with it.

  • Click Actions
  • Click Delete function
  • Click Delete

You’re done! Have fun!

Deploying Multiple Java Applications in a Single Elastic Beanstalk Instance

| Comments

Today I released a Python script called quick-deploy-java that makes it easier to deploy multiple Java applications (WARs) to a single Elastic Beanstalk instance.

Want to give it a shot? Clone the repo and follow along with this tutorial.

Step 1: Create a directory for your application and navigate to that directory

We’ll call this eb-test. We assume that you’ll do this from the same directory that this TUTORIAL.md file is located. If you do not the instructions later will need to incorporate the proper paths.

1
2
$ mkdir eb-test
$ cd eb-test

Step 2: Run eb init to create your application in Elastic Beanstalk

1
$ eb init

I used the following values:

Region – us-east-1

1
2
3
4
5
6
7
8
9
10
11
Select a default region
1) us-east-1 : US East (N. Virginia)
2) us-west-1 : US West (N. California)
3) us-west-2 : US West (Oregon)
4) eu-west-1 : EU (Ireland)
5) eu-central-1 : EU (Frankfurt)
6) ap-southeast-1 : Asia Pacific (Singapore)
7) ap-southeast-2 : Asia Pacific (Sydney)
8) ap-northeast-1 : Asia Pacific (Tokyo)
9) sa-east-1 : South America (Sao Paulo)
(default is 3): 1

Application name – eb-test

1
2
3
Enter Application Name
(default is "eb-test"): 
Application eb-test has been created.

Platform – Tomcat

1
2
3
4
5
6
7
8
9
10
11
12
Select a platform.
1) Node.js
2) PHP
3) Python
4) Ruby
5) Tomcat
6) IIS
7) Docker
8) Multi-container Docker
9) GlassFish
10) Go
(default is 1): 5

Platform version – Tomcat 8 Java 8

1
2
3
4
5
6
7
Select a platform version.
1) Tomcat 8 Java 8
2) Tomcat 7 Java 7
3) Tomcat 7 Java 6
4) Tomcat 7
5) Tomcat 6
(default is 1): 1

SSH – yes

1
2
Do you want to set up SSH for your instances?
(y/n): y

Keypair – existing

I already had a keypair called aws-eb that I use. You can reuse an existing one or create a new one. It shouldn’t matter.

1
2
3
4
Select a keypair.
1) aws-eb
2) [ Create new KeyPair ]
(default is 2): 1

Step 3: Create a testing environment

Since no applications are in this directory it will also launch the sample application. On my system this step took about 4 minutes.

1
$ eb create testing

The output you’ll see will start with something similar to this:

1
2
3
4
5
WARNING: The current directory does not contain any source code. Elastic Beanstalk is launching the sample application instead.
Environment details for: testing
  Application name: eb-test
  Region: us-east-1
  Deployed Version: Sample Application

Once it is done you’ll see this:

1
INFO: Successfully launched environment: testing

Step 4: Validate that the default environment is running

1
$ eb open

You should see a page in your web browser that looks like this:

Default Tomcat 8 Java 8 application

Step 5: Deploy the first toy application by itself

1
2
3
4
5
$ ../quick-deploy.py . ../example-applications/application1/
Building ROOT application...
ROOT application built
Deploying application.  This can take a long time.
Application deployed

This may take several minutes. Once it is complete you can verify that it is running by doing this:

1
$ eb open

In your browser you should see a simple page that says something like this:

1
This is application1 and it prints out a big random number 1555709810405276531

The random number at the end will be different.

At this point you’ve deployed a single application to a single instance. Now we’ll deploy three applications to the same instance.

Step 6: Deploy three applications to the same instance

1
2
3
4
5
6
7
$ ../quick-deploy.py . ../example-applications/application1/ ../example-applications/application1/ 1 ../example-applications/application2/ 2
Building ROOT application...
ROOT application built
Building 1 application...
Building 2 application...
Deploying application.  This can take a long time.
Application deployed

This took about 4 minutes on my system. At this point open the application again:

1
$ eb open

Make sure that the random number has changed. This shows that you’ve got a new instance of the application.

Now add /1 to the URL in your address bar and load the page. You should see the same message but again with another new number. This shows that you have a second copy of the application running at a new path.

Finally replace /1 with /2 in the URL in your address bar and load the page. You should see exactly this message in your browser:

1
This is application2 and it just prints out this message

If everything worked out you now have an Elastic Beanstalk configuration that is running three applications in a single instance. application1 is running at the root and at the 1 path while application1 is running at the 2 path.

Step 7: Terminate your application

Make sure you terminate your application when you’re done.

1
$ eb terminate

If you forget to do this step Amazon will keep your application running and keep billing you! If you want to be extra thorough go into the AWS Elastic Beanstalk console and delete the application completely.

You should also go into S3 and delete any files that are left behind. They don’t take up much space but keeping S3 tidy for when you start making real use of Elastic Beanstalk makes sense.

Deploying Multiple WAR Files on a Single Instance With Elastic Beanstalk’s Command-line Tools

| Comments

UPDATE 2015-07-31: Removed the section about .ebextensions. I have confirmed that this is not needed. This mistake was an artifact of my troubleshooting process.

UPDATE 2015-07-28: I briefly talked to Nick Humrich and Abhishek K Singh on Twitter, pointed them to this article, and it was fixed in less than 24 hours. Nice work! If you update your EB CLI tools to 3.4.7 or greater the bug will be fixed for you and you can ignore the rest of this article.

In June this year AWS added the ability to run multiple WAR files with Elastic Beanstalk in a single EC2 instance. This makes deploying several small applications a lot more cost effective.

In order to do this you need to create a ZIP file containing all of the WAR files you want to deploy and you must include a directory called .ebextensions even if it is empty. In my case I just added and empty file called .ebextensions/README.md to make it happy. Obviously if you use .ebextensions for any kind of customization you won’t need to do anything.

When Elastic Beanstalk sees that you’ve deployed a file like this it treats it differently than a normal bundle. It takes the WAR file called ROOT.war and deploys that as the root application. The rest of the WAR files are deployed in directories derived from their file names. For example, application1.war would be accessed through the /application1 path.

Deploying with eb deploy creates a ZIP file that includes everything in your current directory. The problem is that eb init creates a .gitignore file in your deployment directory. eb deploy picks that up that .gitignore and adds it to the archive. On the EC2 instance that runs your application it sees that your archive doesn’t contain just WAR files and the .ebextensions directory and it treats it like a normal single application deployment. If/when this happens you’ll see that you just get your WAR files placed in the /var/lib/tomcat8/webapps/ROOT directory.

There are two possible fixes for this problem.

Solution #1: You can just delete the .gitignore file. IMHO, this isn’t a great solution since you need to remember to do it for each project.

Solution #2: You can patch the Python code that creates the ZIP archive and tell it to ignore the .gitignore file. To do that you need to modify your ebcli/core/fileoperations.py file. On my system, since I’m not using pip, it was located at ~/.ebvenv/lib/python2.7/site-packages/ebcli/core/fileoperations.py. If you use pip or another package manager it will probably be in a different location.

The function you need to modify is called zip_up_folder. My original version looks like this:

1
2
3
4
5
6
7
8
9
10
11
def zip_up_folder(directory, location, ignore_list=None):
    cwd = os.getcwd()
    try:
        os.chdir(directory)
        io.log_info('Zipping up folder at location: ' + str(os.getcwd()))
        zipf = zipfile.ZipFile(location, 'w', zipfile.ZIP_DEFLATED)
        _zipdir('./', zipf, ignore_list=ignore_list)
        zipf.close()
        LOG.debug('File size: ' + str(os.path.getsize(location)))
    finally:
        os.chdir(cwd)

The code I added to set up the ignore_list is:

1
2
3
    if ignore_list is None:
        ignore_list = []
    ignore_list.append(".gitignore")

So my updated function looks like this:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
def zip_up_folder(directory, location, ignore_list=None):
    cwd = os.getcwd()
    try:
        os.chdir(directory)
        io.log_info('Zipping up folder at location: ' + str(os.getcwd()))
        zipf = zipfile.ZipFile(location, 'w', zipfile.ZIP_DEFLATED)

        if ignore_list is None:
            ignore_list = []
        ignore_list.append(".gitignore")

        _zipdir('./', zipf, ignore_list=ignore_list)
        zipf.close()
        LOG.debug('File size: ' + str(os.path.getsize(location)))
    finally:
        os.chdir(cwd)

After updating that code and doing another eb deploy my set of WARs were all happily running in a single EC2 instance as I’d expect.

How Would You Implement “Cron” on AWS as Inexpensively as Possible?

| Comments

There seems to be a common problem I’ve seen in app developer threads. CloudKit provides tons of functionality to run your app without your own infrastructure but it only runs the APIs they provide. There is an Android syncing API that provides similar things but with the same limitations. Lots of app developers are really cheap because they’re testing the water to see if their app is worth spending money on or not.

The problem: You need to do some simple processing on a data set and give the processed data set to your users.

The solution: Run a scheduled job that processes the data and puts it in S3.

The constraints: You must use AWS. The cost must be under $9 / month (less than a t2.micro) and cheaper is better. You can’t run it on your laptop/Raspberry Pi. It must run at least once per day.

I think it’s a fun exercise to see how this problem can be solved. As I’m learning more and more about AWS there seem to be multiple ways to do it so I’m interested in seeing what other people come up with.

Some possible solutions I’ve seen so far:

How would you do it?

Validating Guava Event Bus Interactions With Mockito

| Comments

Are you using Guava’s event bus in your Java project? Do you need to test and validate interactions with the event bus but are having trouble? Mockito can help you out here with just a few lines of code.

Assume you have two event types. We’ll call them WantedEvent and UnwantedEvent. In your test you can use Mockito to create a mock event bus like this:

1
mockEventBus = mock(EventBus.class);

Then you can use Mockito’s doAnswer functionality to intercept all interactions with the event bus and do something like this:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
doAnswer(new Answer<Void>() {
    @Override
    public Void answer(InvocationOnMock invocationOnMock) throws Throwable {
        Object argument = invocationOnMock.getArguments()[0];

        if (argument instanceof WantedEvent) {
            wantedEventCount++;
        } else if (argument instanceof UnwantedEvent) {
            unwantedEventCount++;
        } else {
            throw new UnsupportedOperationException("This kind of event was not expected");
        }

        return null;
    }
}).when(mockEventBus).post(anyObject());

When you use your event bus in your tests now you’ll be counting the number of times WantedEvent and UnwantedEvent are published (you do need to make those variables accessible outside the scope of this anonymous Answer<Void> block).

You’ll also be throwing exceptions if you see any other kinds of events that you didn’t expect so you can fail immediately if there is additional unwanted behavior going on. Naturally you can leave the else part out and ignore other events completely too.

Using this simple pattern I’ve been able to debug, test, and be confident in the implementation of several projects that use the Guava event bus.

Do you have any tricks or tips related to this article? Is there a better or easier way to do what I’m doing here? Did this get you out of a bind? Post in the comments!

Mockito and ServletInputStreams

| Comments

I was working on a few applications that involve servlets recently and I came across a situation that initially seemed challenging to test with Mockito. I wanted to do something relatively simple which was read a Protobuf sent from a client, turn it into an object, and do some processing on it.

The question is how do you test a servlet that needs to get the input stream from a servlet request?

I found a Stack Overflow post that addresses how to do this with an older version of the ServletInputStream but doing it now requires that you override three additional methods (isFinished, isReady, and setReadListener).

My issue with this is that I don’t want to override those methods because I don’t really know what I want them to do. If I’m mocking something I want to make sure I know when and where it will be used or I want the mocking framework to return default values or throw exceptions so I know where to look when something breaks.

What I landed on was using the thenAnswer method like this:

Mocking a ServletInputStream
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
byte[] myBinaryData = "TEST".getBytes();
ByteArrayInputStream byteArrayInputStream = new ByteArrayInputStream(myBinaryData);
      
mockServletInputStream = mock(ServletInputStream.class);

when(mockServletInputStream.read(Matchers.<byte[]>any(), anyInt(), anyInt())).thenAnswer(new Answer<Integer>() {
    @Override
    public Integer answer(InvocationOnMock invocationOnMock) throws Throwable {
        Object[] args = invocationOnMock.getArguments();
        byte[] output = (byte[]) args[0];
        int offset = (int) args[1];
        int length = (int) args[2];
        return byteArrayInputStream.read(output, offset, length);
    }
});

If you need to ever mock a ServletInputStream feel free to use this code to do it. So far it has worked perfectly for me.

Fixing Javac on Mac OS When Multiple JVMs Are Installed

| Comments

For some reason I decided to install the Java 8 JDK a few days ago when I upgraded to Yosemite. In IntelliJ it isn’t a problem but on the command-line it isn’t so nice. Here’s what I get when I try to use javac:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
$ javac src/com/timmattison/Main.java
warning: /Library/Java/JavaVirtualMachines/jdk1.8.0_25.jdk/Contents/Home/lib/ct.sym(META-INF/sym/rt.jar/java/lang/Object.class): major version 52 is newer than 51, the highest major version supported by this compiler.
It is recommended that the compiler be upgraded.
warning: /Library/Java/JavaVirtualMachines/jdk1.8.0_25.jdk/Contents/Home/lib/ct.sym(META-INF/sym/rt.jar/java/lang/String.class): major version 52 is newer than 51, the highest major version supported by this compiler.
It is recommended that the compiler be upgraded.
/Library/Java/JavaVirtualMachines/jdk1.8.0_25.jdk/Contents/Home/lib/ct.sym(META-INF/sym/rt.jar/java/lang/Object.class): warning: Cannot find annotation method 'value()' in type 'Profile+Annotation': class file for jdk.Profile+Annotation not found
/Library/Java/JavaVirtualMachines/jdk1.8.0_25.jdk/Contents/Home/lib/ct.sym(META-INF/sym/rt.jar/java/lang/String.class): warning: Cannot find annotation method 'value()' in type 'Profile+Annotation'
warning: /Library/Java/JavaVirtualMachines/jdk1.8.0_25.jdk/Contents/Home/lib/ct.sym(META-INF/sym/rt.jar/java/lang/AutoCloseable.class): major version 52 is newer than 51, the highest major version supported by this compiler.
It is recommended that the compiler be upgraded.
/Library/Java/JavaVirtualMachines/jdk1.8.0_25.jdk/Contents/Home/lib/ct.sym(META-INF/sym/rt.jar/java/lang/AutoCloseable.class): warning: Cannot find annotation method 'value()' in type 'Profile+Annotation'
warning: /Library/Java/JavaVirtualMachines/jdk1.8.0_25.jdk/Contents/Home/lib/ct.sym(META-INF/sym/rt.jar/java/lang/System.class): major version 52 is newer than 51, the highest major version supported by this compiler.
It is recommended that the compiler be upgraded.
/Library/Java/JavaVirtualMachines/jdk1.8.0_25.jdk/Contents/Home/lib/ct.sym(META-INF/sym/rt.jar/java/lang/System.class): warning: Cannot find annotation method 'value()' in type 'Profile+Annotation'
warning: /Library/Java/JavaVirtualMachines/jdk1.8.0_25.jdk/Contents/Home/lib/ct.sym(META-INF/sym/rt.jar/java/io/PrintStream.class): major version 52 is newer than 51, the highest major version supported by this compiler.
It is recommended that the compiler be upgraded.
/Library/Java/JavaVirtualMachines/jdk1.8.0_25.jdk/Contents/Home/lib/ct.sym(META-INF/sym/rt.jar/java/io/PrintStream.class): warning: Cannot find annotation method 'value()' in type 'Profile+Annotation'
warning: /Library/Java/JavaVirtualMachines/jdk1.8.0_25.jdk/Contents/Home/lib/ct.sym(META-INF/sym/rt.jar/java/io/FilterOutputStream.class): major version 52 is newer than 51, the highest major version supported by this compiler.
It is recommended that the compiler be upgraded.
/Library/Java/JavaVirtualMachines/jdk1.8.0_25.jdk/Contents/Home/lib/ct.sym(META-INF/sym/rt.jar/java/io/FilterOutputStream.class): warning: Cannot find annotation method 'value()' in type 'Profile+Annotation'
warning: /Library/Java/JavaVirtualMachines/jdk1.8.0_25.jdk/Contents/Home/lib/ct.sym(META-INF/sym/rt.jar/java/io/OutputStream.class): major version 52 is newer than 51, the highest major version supported by this compiler.
It is recommended that the compiler be upgraded.

When I run javac -version I get mostly what I’d expect:

1
2
$ javac -version
javac 1.7.0_45

So why is it trying to use libraries from the Java 8 JDK? Simply because I forgot to set JAVA_HOME. On Mac OS you can quickly fix this by adding the following line to your .bash_profile and starting a new Terminal session:

1
export JAVA_HOME="/Library/Java/JavaVirtualMachines/jdk1.7.0_45.jdk/Contents/Home/"

Of course you should change _45 to reflect the specific version you’re running and validate that the path in the JAVA_HOME variable exists.

Good luck!

When Unicode Goes Wrong in Java

| Comments

NOTE: This is only guaranteed to work with the Sun JVM since this option is “an internal detail of Sun’s implementations”.

UPDATE 2014-11-12 6:22 PM: The real fix is to set the environment variable LANG to en_US.UTF-8 right before you start your JVM.

Is Unicode breaking in your application and you can’t figure out where? Maybe data from HttpClient is coming back mangled, maybe database queries via JDBC are having Unicode data replaced with question marks, maybe your protobufs are getting shredded, but somewhere something is eating your Unicode data and nothing you’ve tried fixes it. Well…

Did you know the JVM itself has a global Unicode setting specified by the -Dfile.encoding option? Most people I talked to didn’t know about it, myself included, when I ran into a Unicode issue on a project. After some great teamwork and research we found this option, set it, and everything started working again.

All we had to do was put -Dfile.encoding=UTF8 in the script that ran our JVM and everything was fixed but that was only a temporary fix. You really need to set LANG to en_US.UTF-8. If you want to play with it I created a test project on Github that is incredibly simple and shows the right and wrong settings and what they do to a simple trademark symbol. Otherwise, try this on your project and see if it fixes the issue.

Good luck!

Using Interfaces in Camel’s Java DSL With Spring

| Comments

When writing some routes with Camel’s Java DSL I came across this exception:

1
Caused by: java.lang.IllegalStateException: No method invocation could be created, no matching method could be found on: null

After a lot of tracing I figured out that it was related to me calling the .bean(...) method with a class that was actually just an interface. What was happening was that Camel wants to instantiate this class, usually using Spring, but cannot do that if it isn’t a concrete implementation.

This proved to be a real problem because I had an interface that had two implementations. One of these implementations is used for debugging and the other is used for production. I didn’t want to have to manually select which one I was using in my code because that’s Spring’s job so I came up with a way to do it.

For the complete background here’s what my interface looks like:

The interface
1
2
3
4
import org.apache.camel.Processor;

public interface ProtobufToWire extends Processor {
}

This converts a Protobuf to our “wire” format. That format could be the native protobuf binary format or JSON. I implement this empty interface in two classes called ProtobufToBinary and ProtobufToJson and I want to use the JSON one only for debugging.

To be clear doing this always fails with the exception I listed above:

A route that always fails
1
from(SOME_URI).bean(ProtobufToWire.class);

To fix this I added this to my Java-based Spring config:

Getting an instance of ProtobufToWire
1
2
3
4
@Bean
public ProtobufToWire protobufToWire() {
    return new ProtobufToBinary();
}

Now because, I believe, that Camel’s bean(...) method doesn’t look up the beans with Spring this still fails. What I needed to where I am defining my routes is this:

Finally, how to get Camel to instantiate the right type
1
2
3
4
5
6
7
    @Autowired
    private ProtobufToWire protobufToWire;

    @Override
    public void configure() {
      from(SOME_URI).bean(protobufToWire.getClass());
    }

What I’m doing here is getting Spring to autowire an instance of that interface into a private variable and then asking it for its real concrete type. Part of me says that I shouldn’t have to do this but this is what works for me.

Did this help you out? Do you have a better way to do it? Post in the comments!

Activating U2F on a Yubikey Neo on Mac OS

| Comments

I just got my Yubikey Neo with U2F support and I felt like the documentation on how to get it up and running was a bit hard to find. If you are having trouble getting started with U2F these few quick steps will help you get through it.

Step 0: Download and install the Yubikey Neo Manager application. This is NOT the Yubikey Personalization Tool! The Yubikey Personalization Tool does not support enabling U2F yet.

Step 1: Open the Yubikey Neo Manager with your Yubikey installed and click Change connection mode [OTP] Yubikey Neo Manager main screen

Step 2: On the Change connection mode check the U2F box to change the setting from OTP to U2F and click OK. Yubikey Neo Manager main screen Yubikey Neo Manager main screen

The application will now prompt you to remove your device. You can remove it and plug it back in again. Close the Yubikey Neo Manager application.

Step 3: Open Chrome and install the FIDO U2F (Universal 2nd Factor) extension from the Chrome web store.

Step 4: Register on Yubico’s U2F demo page and you’re done.

Now you can log in on the demo page and other sites that support U2F.