Thursday, February 12, 2015

How long did I sleep last night: using Cordova, HealthKit, and JavaScript, and a handful of Promises

Now that we can know how much activity I did yesterday, we can look at whether I am getting enough sleep.

A quick look at the Apple documentation lets us see that HKCategoryTypeIdentifierSleepAnalysis is the right measure to be using. This is a subtype of SampleType so we can use the querySampleType function.

First of all here is the same boilerplate code, with the minor modification to allow for access to the sleep data rather than the step data we used before. You could of course combine the two.

var wearable = {

    avaliable: _.once(function() {

        return new Promise(function(resolve, reject) {
        
            if (window.plugins.healthkit) {
                window.plugins.healthkit.available(
                        function() {
                            console.log("Healthkit is avaliable");
                            resolve(true);
                        },
                        function() {
                            console.log("Healthkit is not avaliable");
                            reject(Error(false));
                        }
                );
            } else {
                reject(Error("HealthKit Not Available"));
            }
        });
    }),


    getHealthKit : _.once(function() {

        return wearable.avaliable().then(function() {

            return new Promise(function(resolve, reject) {
                window.plugins.healthkit.requestAuthorization(
                        {
                            'readTypes': ['HKCategoryTypeIdentifierSleepAnalysis'],
                            'writeTypes': []
                        },
                function() {
                    console.log("HealthKit authorisation accepted");
                    resolve(window.plugins.healthkit);
                },
                function() {
                    reject(Error("HealthKit authorisation rejected"));
                });
            });
        });
    }),

    ...

}



Then it is s simple matter of querying HeathKit with a useful date range. Note I use limit and ascending to access the last data point - it is possible that different tools will report multiple entires for a single night. (In particular new parents) I need to do a little bit more investigation here.

wearable = {

    ... 

    querySleep: function() {

        wearable.getHealthKit().then(function(healthkit) {

                var startDate = moment().subtract('d', 1).toDate();

                healthkit.querySampleType({
                        'startDate': startDate,
                        'endDate': moment().toDate(),
                        'sampleType': "HKCategoryTypeIdentifierSleepAnalysis",
                        'ascending': "NO",
                        'limit': 1
                    },
                    function(value) {
                        console.log("Success for runing sleep query");

                        // Debug output for the momment
                        value.forEach(function(next) {

                            console.dir(next);
                            var measure = next.value === 1 ? "Sleeping" : "In Bed";
                            var minutesSleep = moment(next.endDate).diff(next.startDate, "minutes");
                            var hoursSleep = moment(next.endDate).diff(next.startDate, "hours");
                            console.log("Entry got " + minutesSleep + " minutes " + measure);
                            console.log("Entry got " + hoursSleep + " hours " + measure);

                        });

                        // Use data in some way
                    });

            }
        }
    },
}


Finally it would be really useful to be able to be told when data is added to HealthKit by another app for example the client app for a wearable. You can do this using the monitorSampleType function. Ideally this would code would then use something called an anchored query so you only access the most recently added data; but I haven't had time to implement this yet so we simply call querySleep().

Now making sure you app wakes up in the background to receive this data is a whole other blog....

var wearable = {

    ...

    monitorSleep: _.once(function() {

        console.log("Starting to monitor sleep");


        wearable.getHealthKit().then(function(healthkit) {
            healthkit.monitorSampleType({
                    'sampleType': "HKCategoryTypeIdentifierSleepAnalysis"
                },
                _.debounce(function(value) {
                    console.log("Sleep data has been updated, lets see if anything interesting has been added");
                    wearable.querySleep();
                }, 2000),
                function() {
                    console.log("Failed to monitor sample data");
                    console.dir(arguments);
                });

        });
    })
}




Tuesday, February 10, 2015

Per client cookie handling with Jersey

A lot of REST services will use cookies as part of the authentication / authorisation scheme. This is a problem because by default the old Jersey client will use the singleton CookieHandler.getDefault which is most cases will be null and if not null will not likely work in a multithreaded server environment. (This is because in the background the default Jersey client will use URL.openConnection)

Now you can work around this by using the Apache HTTP Client adapter for Jersey; but this is not always available. So if you want to use the Jersey client with cookies in a server environment you need to do a little bit of reflection to ensure you use your own private cookie jar.

final CookieHandler ch = new CookieManager();
    
    Client client = new Client(new URLConnectionClientHandler(
       new HttpURLConnectionFactory() {

        @Override
        public HttpURLConnection getHttpURLConnection(URL uRL) throws IOException {
            HttpURLConnection connect = (HttpURLConnection) uRL.openConnection();
        
            try {
                
                Object toModify = connect;
                
                if (!(toModify instanceof sun.net.www.protocol.http.HttpURLConnection)) {
        
                    Field delegateField = connect.getClass().getDeclaredField("delegate");
                    delegateField.setAccessible(true);
                    toModify = MethodHandles.lookup().unreflectGetter(delegateField)
                        .bindTo(toModify).invoke();
                    
                }
        
                Field cookieField = sun.net.www.protocol.http.HttpURLConnection.class.getDeclaredField("cookieHandler");
                cookieField.setAccessible(true);
                MethodHandle mh = MethodHandles.lookup().unreflectSetter(cookieField);
                mh.bindTo(toModify).invoke(ch);
            } catch (Throwable e) {
                e.printStackTrace();
            }
        
            return connect;

        }
    }));
    

This will only work if your environment is using the internal implementation of sun.net.www.protocol.http.HttpURLConnection that comes with the JDK. This appears to be the case for modern versions of WLS.

For JAX-RS 2.0 you can do a similar change using Jersey 2.x specific ClientConfig class and HttpUrlConnectorProvider.

final CookieHandler ch = new CookieManager();


    Client client =
        ClientBuilder.newClient(new ClientConfig().connectorProvider(new HttpUrlConnectorProvider().connectionFactory(new HttpUrlConnectorProvider.ConnectionFactory() {
            @Override
            public HttpURLConnection getConnection(URL uRL) throws IOException {
                HttpURLConnection connect = (HttpURLConnection) uRL.openConnection();
            
                try {
                    
                    Object toModify = connect;
                    
                    if (!(toModify instanceof sun.net.www.protocol.http.HttpURLConnection)) {
            
                        Field delegateField = connect.getClass().getDeclaredField("delegate");
                        delegateField.setAccessible(true);
                        toModify = MethodHandles.lookup().unreflectGetter(delegateField)
                            .bindTo(toModify).invoke();
                        
                    }
            
                    Field cookieField = sun.net.www.protocol.http.HttpURLConnection.class.getDeclaredField("cookieHandler");
                    cookieField.setAccessible(true);
                    MethodHandle mh = MethodHandles.lookup().unreflectSetter(cookieField);
                    mh.bindTo(toModify).invoke(ch);
                } catch (Throwable e) {
                    e.printStackTrace();
                }
            
                return connect;

            }
        })));



Update 11th Feb 2015: It seems in some cases, in particular using https, I have seen the HttpURLConnection wrapped in another class, to work around this just use reflection to access the value of the delegate field. I have updated the code examples to reflect this issue.

Sunday, February 1, 2015

How many steps did I walk yesterday: using Cordova, HealthKit, and JavaScript, and a handful of Promises

As people who know me might know I have been playing around with various wearables for some years now, starting with a Nike Fuel Band, and also trying out products from Misfit and UP along the way. I have mostly just watched with interest as my exercise levels, and indeed sleep now we have a little one in the house, has gone up and down over time.

More recently with the arrival of HealthKit there comes a standard method of accessing this information from multiple products on the iPhone, I needed to brush up on my JavaScript for work so I decided to have a play in Cordova to build something in my spare time. One of the cool things about that Cordova community when compared to other app development frameworks is the large and healthy plugin community - and quick google and I found what I needed to support HealthKit HealthKit. This took only a small amount of patching from me to get at the information I wanted. In this blog I will just look at number of steps per day as a proxy for activity.

I am going to assume the rest of the Cordova App is in place and start with a simple wearable object which exposes a promise that allows you to check whether HealthKit is available on this platform, you might have iOS 8 but HealthKit is iPhone only. Remember that most of Cordova is interacting with a bunch of asynchronous native systems so you have to make sure you consider this as you work with it. Hence the liberal use of promises. (If you are wanting to support < iOS 8 then you will need to use a different promise library / shim) In this case I am also using the underscorejs library to create a promise who's result won't be calculated until it is invoked for the first time.

var wearable = {

    avaliable: _.once(function() {

        return new Promise(function(resolve, reject) {
        
            if (window.plugins.healthkit) {
                window.plugins.healthkit.available(
                        function() {
                            console.log("Healthkit is avaliable");
                            resolve(true);
                        },
                        function() {
                            console.log("Healthkit is not avaliable");
                            reject(Error(false));
                        }
                );
            } else {
                reject(Error("HealthKit Not Available"));
            }
        });
    }),

...

}

The next stage of the code, once you are sure that the system is available, is to get verify that you can access the data you want to access. Now Apple is clear in there developer guide that you should only request the data you might want to access once. So no asking for step data one time then heath rate data in a different part of the UI. The Cordova HealthKit Plugin will just return with a success if those values are already authorised, so we will use another promise as a gateway to actually working with the data to make one consistent entry point for authorisation.

var wearable = {

...


    getHealthKit : _.once(function() {

        return wearable.avaliable().then(function() {

            return new Promise(function(resolve, reject) {
                window.plugins.healthkit.requestAuthorization(
                        {
                            'readTypes': ['HKQuantityTypeIdentifierStepCount'],
                            'writeTypes': []
                        },
                function() {
                    console.log("HealthKit authorisation accepted");
                    resolve(window.plugins.healthkit);
                },
                function() {
                    reject(Error("HealthKit authorisation rejected"));
                });
            });
        });
    }),

...

}

Finally now we have all the ground work in place, lets put in place a simple method to work out how many steps were recorded yesterday. I am then using special function called sumQuantityType that performs some rather clever statistical leg work for you. Consider if you have a iPhone 6 along with step monitoring wrist band of some kind like a Apple Watch, in HealthKit you will have two lots of data that inconsistently overlaps. HealthKit provides methods to deal with this, more information can be found in the WWDC presentation from 2014, but basically it works out for each time segment which data source(s) to use to get the value for the time range requested.

var wearable = {

...


    querySteps : function() {
        
        return wearable.getHealthKit().then(function (healthkit)
        {
            return new Promise(function(resolve, reject) {

                var m = moment().startOf('day');
                var endDate = m.toDate();
                var startDate = moment(m).subtract('d', 1).toDate();

                healthkit.sumQuantityType({
                        'startDate': startDate,
                        'endDate': endDate,
                        'sampleType': "HKQuantityTypeIdentifierStepCount"
                    },
                    function(value)
                    {
                        resolve(value)
                    },
                    function()
                    {
                        reject(Error("Problem queuing steps"));
                    });
        });
    }

}

Now we have this object, we can access the data and show it to the user. Note that the promise objects chain so the catch function will tell you if something went wrong at any stage of the process, if you had tried to wire this up with event handlers you would have had an ungodly mess of code and callbacks.

wearable.querySteps()
       .then(function(value){
           console.log("Number of steps "  + value);
           ... update UI
       })
       .catch(function (error) {
           ... update UI
       };


This is of course just the start, there is a lot more interesting stuff you can get from this API; but that is for another day. I like HealthKit because unlike GoogleFit all your health data is stored locally by default. There is a lot of potential here for doing interesting stuff.

Finally a quick shout out of Eddy Verbruggen who created HealthKit plugin along with a whole host of other interesting libraries.

I am going to walk around for a bit now, gotta meet those targets. :-)

Monday, November 17, 2014

But that's impossible, or finding out that the JIT has broken your code.

Every now and then you look at some code and think that it cannot be possibly be wrong. Once you have ruled out a simple programmer screw up / enemy action in code (Make sure you read Java Puzzlers or similar) or a concurrency issue (Read Java Concurrency or go on Dr Heniz excellent course) you should sit back and take a few days and then starting thinking about whether the JDK is indeed out to get you. I haven't seen one in the wild in my 18 odd years as a Java programmer so it kinda took me by surprise.

If you are running against JDK 8 in a large scale Swing application you might eventually see the following exception, lots of lots of times. (Unless you forgot the lesson learned in my previous blog in your logging code, in which case you might see lots of ArrayOfOutBoundsException)

Caused by: java.lang.NullPointerException 
    at javax.swing.text.GlyphView.getBreakSpot(GlyphView.java:799) 
    at javax.swing.text.GlyphView.getBreakWeight(GlyphView.java:724) 
    at javax.swing.text.FlowView$LogicalView.getPreferredSpan(FlowView.java:733) 
    at javax.swing.text.FlowView.calculateMinorAxisRequirements(FlowView.java:233) 
    at javax.swing.text.ParagraphView.calculateMinorAxisRequirements(ParagraphView.java:717) 
    at javax.swing.text.BoxView.checkRequests(BoxView.java:935) 
    at javax.swing.text.BoxView.getMinimumSpan(BoxView.java:568) 
    at javax.swing.text.BoxView.calculateMinorAxisRequirements(BoxView.java:903) 
    at javax.swing.text.BoxView.checkRequests(BoxView.java:935) 
    at javax.swing.text.BoxView.setSpanOnAxis(BoxView.java:343) 
    at javax.swing.text.BoxView.layout(BoxView.java:708) 
    at javax.swing.text.BoxView.setSize(BoxView.java:397) 
    ...

This error is particularly insidious because it takes around ten minutes to show itself and sometimes not at all. If you look at the code for this class the line in question, starts with "startsFrom = break", only accesses two local variables both of which have been previously referenced in the method.

            Segment s = getText(pstart, pend); 
            s.first(); 
            BreakIterator breaker = getBreaker(); 
            breaker.setText(s); 

            // Backward search should start from end+1 unless there's NO end+1 
            int startFrom = end + (pend > end ? 1 : 0); 
            for (;;) { 
                startFrom = breaker.preceding(s.offset + (startFrom - pstart))   
                          + (pstart - s.offset); 
                if (startFrom > start) { 
                    // The break spot is within the view 
                    bs[ix++] = startFrom; 
                } else { 
                    break; 
                } 
            } 

The most direct way to rule out a JIT error is to disable compilation for just this single method, here is an example; but you can fine more in the documentation for the command line java tool.

javaThing -XX:CompileCommand=exclude,javax/swing/text/GlyphView,getBreakSpot

When this parameter is added, the problem goes away. - since we have ruled out enemy action by code or a concurrency issue we can be more sure this is a JIT issue. Now as part of the bug logging for this I output diagnostics for this single method, and found out that the problem, didn't occur until the method was JITted for a fifth time.

javaThing -XX:CompileCommand=print,javax/swing/text/GlyphView,getBreakSpot

Here is some diagnostic output seen with the above command:

Compiled method (c2)  914078 33142       4       javax.swing.text.GlyphView::getBreakSpot (247 bytes)
 total in heap  [0x00002aaab0749e10,0x00002aaab0750fe0] = 29136
 relocation     [0x00002aaab0749f38,0x00002aaab074a1e8] = 688
 constants      [0x00002aaab074a200,0x00002aaab074a2a0] = 160
 main code      [0x00002aaab074a2a0,0x00002aaab074cde0] = 11072
 stub code      [0x00002aaab074cde0,0x00002aaab074ce40] = 96
 oops           [0x00002aaab074ce40,0x00002aaab074ce58] = 24
 metadata       [0x00002aaab074ce58,0x00002aaab074d058] = 512
 scopes data    [0x00002aaab074d058,0x00002aaab074ea20] = 6600
 scopes pcs     [0x00002aaab074ea20,0x00002aaab0750c50] = 8752
 dependencies   [0x00002aaab0750c50,0x00002aaab0750c80] = 48
 handler table  [0x00002aaab0750c80,0x00002aaab0750e90] = 528
 nul chk table  [0x00002aaab0750e90,0x00002aaab0750fe0] = 336
OopMapSet contains 113 OopMaps
#0 
OopMap{[8]=Oop [32]=Oop [40]=Oop off=892}
#1 
OopMap{[32]=Oop [40]=Oop off=960}
#2 
OopMap{[32]=Oop [40]=Oop off=980}
#3 
OopMap{[32]=Oop [40]=Oop [48]=Oop off=1048}
#4 
OopMap{[32]=Oop [40]=Oop [48]=Oop off=1084}
#5 
OopMap{[0]=Oop [24]=Oop [48]=Oop [56]=Oop [80]=Oop off=2500}
#6 
OopMap{rbx=Oop rdi=Oop [32]=Oop [40]=Oop [112]=Oop off=2533}
#7 
OopMap{rbx=Oop rdi=Oop r14=Oop [32]=Oop [112]=Oop off=3081}
#8 
OopMap{rbx=Oop rdi=Oop r14=Oop [32]=Oop [40]=Oop [112]=Oop off=3190}
#9 
OopMap{[8]=Oop [32]=Oop [40]=Oop off=4408}
#10 
OopMap{[32]=Oop [40]=Oop [48]=Oop off=4640}
#11 
OopMap{rbp=Oop [16]=Oop [24]=Oop [40]=Oop [64]=Oop off=5232}
#12 
OopMap{rbp=Oop [0]=NarrowOop [32]=Oop off=5364}
#13 
OopMap{[32]=Oop [40]=Oop [48]=Oop off=5408}
#14 
OopMap{rbp=Oop [32]=Oop [40]=Oop [48]=Oop off=5436}
#15 
OopMap{rbp=Oop [32]=Oop [40]=Oop [48]=Oop off=5468}
#16 
OopMap{rbp=Oop [32]=Oop [40]=Oop [48]=Oop off=5524}
#17 
OopMap{rbp=Oop [32]=Oop [40]=Oop [48]=Oop [88]=Oop off=5552}
#18 
OopMap{[32]=Oop [40]=Oop [48]=Oop [64]=Oop [72]=Derived_oop_[64] [112]=Oop off=5608}
#19 
OopMap{[8]=Oop [32]=Oop off=5680}
#20 
OopMap{rbp=Oop off=5720}
#21 
OopMap{rbp=Oop off=5752}
#22 
OopMap{rbp=Oop [24]=NarrowOop [28]=NarrowOop [32]=Oop [40]=Oop [48]=Oop [56]=Oop [64]=Oop [88]=Oop off=5812}
#23 
OopMap{rbp=Oop [16]=Oop [24]=Oop [40]=Oop [64]=Oop [88]=Oop off=5960}
#24 
OopMap{[0]=Oop [24]=Oop [48]=Oop [56]=Oop [72]=Oop [88]=NarrowOop off=6056}
#25 
OopMap{[40]=Oop off=6088}
#26 
OopMap{[0]=Oop off=6120}
#27 
OopMap{[8]=Oop [24]=Oop [56]=Oop [72]=Oop [112]=Oop off=6216}
#28 
OopMap{[0]=Oop [32]=NarrowOop [40]=Oop off=6284}
#29 
OopMap{rbp=Oop [16]=Oop [40]=Oop [64]=Oop [112]=Oop off=6384}
#30 
OopMap{[0]=Oop off=6412}
#31 
OopMap{[0]=Oop [16]=Oop [32]=NarrowOop [40]=Oop [48]=Oop off=6488}
#32 
OopMap{rbp=Oop [16]=Oop [40]=Oop [48]=Oop off=6560}
#33 
OopMap{[32]=Oop [40]=Oop [48]=Oop [64]=Oop [112]=Oop off=6608}
#34 
OopMap{[8]=Oop [28]=NarrowOop [32]=Oop [40]=Oop [48]=Oop off=6768}
#35 
OopMap{rbp=NarrowOop [0]=Oop [16]=Oop [32]=Oop [40]=NarrowOop off=6860}
#36 
OopMap{[0]=Oop [16]=Oop [32]=NarrowOop [40]=Oop [48]=Oop off=6988}
#37 
OopMap{rbp=Oop [32]=Oop off=7024}
#38 
OopMap{rbp=NarrowOop [0]=Oop [24]=Oop [32]=Oop off=7260}
#39 
OopMap{rbp=NarrowOop [0]=Oop [24]=Oop [32]=Oop off=7344}
#40 
OopMap{rbp=Oop [16]=Oop [24]=Oop [40]=Oop [60]=NarrowOop [64]=Oop off=7452}
#41 
OopMap{rbp=NarrowOop [32]=Oop off=7476}
#42 
OopMap{rbp=NarrowOop [0]=Oop off=7524}
#43 
OopMap{[32]=Oop [40]=Oop [48]=Oop off=7588}
#44 
OopMap{[32]=Oop [40]=Oop [48]=Oop off=7616}
#45 
OopMap{[32]=Oop [40]=Oop [48]=Oop off=7632}
#46 
OopMap{rbp=NarrowOop [32]=Oop off=7676}
#47 
OopMap{rbp=NarrowOop [0]=Oop off=7724}
#48 
OopMap{[0]=Oop [16]=Oop [28]=NarrowOop [40]=Oop [48]=Oop [56]=NarrowOop [64]=Oop off=7868}
#49 
OopMap{[8]=Oop [28]=NarrowOop [32]=Oop [40]=Oop [48]=Oop [56]=Oop off=7916}
#50 
OopMap{rbp=Oop [16]=Oop [24]=NarrowOop off=8016}
#51 
OopMap{rbp=Oop [16]=Oop [28]=NarrowOop off=8080}
#52 
OopMap{rbp=NarrowOop [0]=Oop [24]=Oop [32]=Oop off=8152}
#53 
OopMap{rbp=Oop [8]=NarrowOop off=8212}
#54 
OopMap{rbp=NarrowOop [32]=Oop off=8236}
#55 
OopMap{rbp=Oop [16]=NarrowOop off=8272}
#56 
OopMap{rbp=NarrowOop [0]=Oop off=8320}
#57 
OopMap{rbp=Oop [12]=NarrowOop off=8360}
#58 
OopMap{rbp=NarrowOop [32]=Oop off=8400}
#59 
OopMap{rbp=Oop [12]=NarrowOop off=8460}
#60 
OopMap{rbp=NarrowOop [0]=Oop off=8508}
#61 
OopMap{rbp=Oop [24]=NarrowOop [40]=Oop off=8572}
#62 
OopMap{rbp=Oop off=8600}
#63 
OopMap{rbp=Oop [8]=Oop [28]=NarrowOop off=8640}
#64 
OopMap{rbp=Oop [8]=Oop [20]=NarrowOop [112]=Oop off=8704}
#65 
OopMap{rbp=Oop [16]=Oop [24]=Oop [48]=Oop off=8788}
#66 
OopMap{rbp=Oop [16]=Oop [24]=Oop [40]=Oop [64]=Oop off=8912}
#67 
OopMap{rbp=Oop [16]=Oop [24]=Oop [40]=Oop [64]=Oop off=9036}
#68 
OopMap{rbp=Oop [16]=Oop [24]=Oop [40]=Oop [64]=Oop off=9160}
#69 
OopMap{rbp=Oop [16]=Oop [24]=Oop [40]=Oop [64]=Oop off=9284}
#70 
OopMap{rbp=Oop [16]=Oop [24]=Oop [40]=Oop [64]=Oop off=9408}
#71 
OopMap{rbp=Oop [16]=Oop [24]=Oop [40]=Oop [64]=Oop off=9532}
#72 
OopMap{off=9556}
#73 
OopMap{off=9580}
#74 
OopMap{off=9604}
#75 
OopMap{[112]=Oop off=9628}
#76 
OopMap{rbp=Oop [8]=Oop [24]=Oop [32]=NarrowOop off=9696}
#77 
OopMap{rbp=Oop [8]=Oop [24]=NarrowOop off=9760}
#78 
OopMap{off=9784}
#79 
OopMap{off=9812}
#80 
OopMap{off=9836}
#81 
OopMap{off=9860}
#82 
OopMap{off=9884}
#83 
OopMap{off=9908}
#84 
OopMap{off=9932}
#85 
OopMap{off=9956}
#86 
OopMap{off=9980}
#87 
OopMap{off=10004}
#88 
OopMap{off=10028}
#89 
OopMap{rbp=Oop [16]=Oop [28]=NarrowOop off=10092}
#90 
OopMap{rbp=Oop [16]=Oop [24]=Oop [48]=Oop off=10176}
#91 
OopMap{off=10200}
#92 
OopMap{off=10224}
#93 
OopMap{off=10248}
#94 
OopMap{off=10272}
#95 
OopMap{off=10296}
#96 
OopMap{off=10320}
#97 
OopMap{off=10344}
#98 
OopMap{off=10368}
#99 
OopMap{off=10392}
#100 
OopMap{off=10416}
#101 
OopMap{off=10440}
#102 
OopMap{off=10464}
#103 
OopMap{off=10488}
#104 
OopMap{off=10512}
#105 
OopMap{off=10536}
#106 
OopMap{off=10560}
#107 
OopMap{off=10584}
#108 
OopMap{off=10608}
#109 
OopMap{off=10632}
#110 
OopMap{off=10656}
#111 
OopMap{off=10680}
#112 
OopMap{off=11028}
java.lang.NullPointerException
 at javax.swing.text.GlyphView.getBreakSpot(GlyphView.java:799)
 at javax.swing.text.GlyphView.getBreakWeight(GlyphView.java:724)
 at javax.swing.text.html.InlineView.getBreakWeight(InlineView.java:150)
 at javax.swing.text.FlowView$LogicalView.getPreferredSpan(FlowView.java:733)
 at javax.swing.text.FlowView.calculateMinorAxisRequirements(FlowView.java:233)
 at javax.swing.text.ParagraphView.calculateMinorAxisRequirements(ParagraphView.java:717)
 at javax.swing.text.html.ParagraphView.calculateMinorAxisRequirements(ParagraphView.java:157)
 at javax.swing.text.BoxView.checkRequests(BoxView.java:935)
 at javax.swing.text.BoxView.getMinimumSpan(BoxView.java:568)
 at javax.swing.text.html.ParagraphView.getMinimumSpan(ParagraphView.java:270)
 at javax.swing.text.BoxView.calculateMinorAxisRequirements(BoxView.java:903)

Now I am still working with the JDK team for a fix for this one; but I do feel I have discovered a useful set of tools for providing some evidence that the JIT compiler is causing my bad day. And more importantly I have a workaround so I can run my tests until this is resolved.

Thursday, November 6, 2014

Missing stack traces for repeated exceptions (reminder)

A long while ago a optimisation was added to the JVM so that if the same exception is thrown again and again and again a single instance of the Exception is created without the stack trace filled in in order to increase performance. This is an excellent idea unless you are trying to diagnose a problem and you have missed the original error.

If you forgot about this optimisation you send the afternoon looking at the following log output and weeping slightly. (In my defence I have a little one in the house hence the fuzzy brain and lack of blogging action this year)

java.lang.ArrayIndexOutOfBoundsException
java.lang.ArrayIndexOutOfBoundsException
java.lang.ArrayIndexOutOfBoundsException
java.lang.ArrayIndexOutOfBoundsException
java.lang.ArrayIndexOutOfBoundsException
java.lang.ArrayIndexOutOfBoundsException
java.lang.ArrayIndexOutOfBoundsException
java.lang.ArrayIndexOutOfBoundsException
java.lang.ArrayIndexOutOfBoundsException
java.lang.ArrayIndexOutOfBoundsException
java.lang.ArrayIndexOutOfBoundsException
java.lang.ArrayIndexOutOfBoundsException
java.lang.ArrayIndexOutOfBoundsException
java.lang.ArrayIndexOutOfBoundsException
java.lang.ArrayIndexOutOfBoundsException
java.lang.ArrayIndexOutOfBoundsException
...
java.lang.Exception: Uncaught exceptions during test
 at oracle.jdevstudio-testware-tests.level0.testADFcMATS(/.../work/mw9111/jdeveloper/jdev/extensions/oracle.jdevstudio-testware-tests/abbot/common-adf/CreateNewCustomAppAndProjectWithName.xml:25)
 at oracle.jdevstudio-testware-tests.level0.testADFcMATS(/.../work/mw9111/jdeveloper/jdev/extensions/oracle.jdevstudio-testware-tests/abbot/level0/testADFcMATS.xml:94)
 at oracle.abbot.JDevScriptFixture.runTest(JDevScriptFixture.java:555)
 at junit.framework.TestCase.runBare(TestCase.java:134)
 at junit.framework.TestResult$1.protect(TestResult.java:110)
 at junit.framework.TestResult.runProtected(TestResult.java:128)
 at junit.framework.TestResult.run(TestResult.java:113)
 at junit.framework.TestCase.run(TestCase.java:124)
 at junit.framework.TestSuite.runTest(TestSuite.java:243)
 at junit.framework.TestSuite.run(TestSuite.java:238)
 at junit.textui.TestRunner.doRun(TestRunner.java:116)
 at junit.textui.TestRunner.doRun(TestRunner.java:109)
 at oracle.abbot.AbbotRunner.run(AbbotRunner.java:614)
 at oracle.abbot.AbbotAddin$IdeAbbotRunner.run(AbbotAddin.java:634)
 at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.ArrayIndexOutOfBoundsException

[Crickets]

It turns out that you can turn off this optimisation with a simple flag:

java -XX:-OmitStackTraceInFastThrow ....

In my particular case the actual exception causing this trouble was a NPE from the GlyphView code in JDK8. (One that is being caused by a glitch in hotspot it seems) But that in turn was causing the AIOOBE in some logging code clouding the issue even more.

This in particular is a good flag to add by default when running your automated tests, particularly in combination with the stack trace length override I have talked about before.

Wednesday, May 21, 2014

Reading and writing JAX-RS Link objects

JAX-RS contains a rather nice handy representation of the a Link that can be serialised with and adapter into XML and JSON, unfortunately there is a bug in the JAX-RS spec that means that the standard adapter provided is missing the setters required to deserialise the Link later on.

This isn't a problem if you are using the JAX-B RI as it appears to be more relaxed than the standard; but it will be a problem for other implementations. There is a further bug if you are using MOXy, aka EclipseLink, to produce either JSON or XML that it will fail and just call toString() because it doesn't like the type adapter being an inner class of the class that is being adapted. (Bug TBC)


Direction JAXB RI + JAX-RS Adapter MOXy + JAX-RS Adapter MOXy + Outer Adapter MOXy + Outer Adapter + Setters MOXy + Inner Adapter (Different class)
Marshalling Yes No Yes Yes Yes
Unmarshalling Yes No No Yes No


The workaround is to create a new adapter as a top level class in order to replace the one provided by the standard. Just quickly it would look something like this:

import java.util.HashMap;
import java.util.Map;

import javax.ws.rs.core.Link;

import javax.xml.bind.annotation.adapters.XmlAdapter;
import javax.xml.namespace.QName;

public class LinkAdapter
    extends XmlAdapter {

    public LinkAdapter() {
    }

    public Link unmarshal(LinkJaxb p1) {
        
        Link.Builder builder = Link.fromUri(p1.getUri());
        for (Map.Entry<QName, Object> entry : p1.getParams().entrySet()) {
            builder.param(entry.getKey().getLocalPart(), entry.getValue().toString());
        }
        return builder.build();
    }

    public LinkJaxb marshal(Link p1) {
        
        Map<QName, Object> params = new HashMap<>();
        for (Map.Entry<String,String> entry : p1.getParams().entrySet()) {
            params.put(new QName("", entry.getKey()), entry.getValue());
        }
        return new LinkJaxb(p1.getUri(), params);
    }
}

With a simple POJO to go with it.

import java.net.URI;

import java.util.HashMap;
import java.util.Map;

import javax.xml.bind.annotation.XmlAnyAttribute;
import javax.xml.bind.annotation.XmlAttribute;
import javax.xml.namespace.QName;

public class LinkJaxb  {

    private URI uri;
    private Map<QName, Object> params;


    public LinkJaxb() {
        this (null, null);
    }

    public LinkJaxb(URI uri) {
        this(uri, null);
    }

    public LinkJaxb(URI uri, Map<QName, Object> map) {
        
        this.uri = uri;
        this.params = map!=null ? map : new HashMap<QName, Object>();
        
    }



    @XmlAttribute(name = "href")
    public URI getUri() { 
        return uri;
    }

    @XmlAnyAttribute
    public Map<QName, Object> getParams() { 
        return params;
    }

    public void setUri(URI uri) {
        this.uri = uri;
    }

    public void setParams(Map<QName, Object> params) {
        this.params = params;
    }

}


With this you can read and write the Link objects to and from XML, and JSON with MOXy, to your hearts content.

Tuesday, May 20, 2014

Declarative Linking in Jersey 2.9 and up

A couple of weeks ago A couple of months ago I was looking how I was going to engineers new REST API for an Oracle Cloud project. Once of the things I had planned to do was to use the declarative link injection created in Jersey 1.x by Marc Hadley. Sadly this hadn't been forwarded ported yet, so a quick chat to the project lead and I took on the small medium sized job of bringing the code up to date.

One of the things that has changed in the new version is that in JAX-RS 2.0 there is a Link object so rather than being able to only inject String and URI you can also inject the correct rel attributes. This has means that the existing annotations coded by Marc have been merged into once simple set of annotations for both Link headers and for injected properties.

This functionality is available now, along with a simple example. The original version of the feature that I committed has some serious limitations that are described later, you will need a version of Jersey post 2.8 or you can build a 2.9-SNAPSHOT image that contains my changes currently to implement the example in this blog.

This blog looks at using this new API to provide simple injection for a collections API. One of the common patterns in RESTful services, in particular those based on JSON, is to have an array of structural links at the top level of the structure. For the purposes of this blog I am going to follow the form of the Collection+JSON hypermedia type.

{ "collection" :
  {
    "version" : "1.0",
    "href" : "http://example.org/friends/?offset=10&limit=10",
    
    "links" : [
      {"rel" : "create", "href" : "http://example.org/friends/"}
      {"rel" : "next", "href" : "http://example.org/friends/?offset=20&limit=10"}
      {"rel" : "previous", "href" : "http://example.org/friends/?offset=0&limit=10"}
    ],
   
    "items" : [
       ...
    ]
  }
}

So I can inject the links in the following form, not there is a bunch of boiler plate missing here for clarity. This is not the tidiest code; but in a later cycle it should be possible to simply them somewhat. The design currently uses EL to access properties - this has the advantage of making it possible to write back values as you can represent properties. I can understand it is disliked by some; but I am not sure if I see any value in moving to JavaScript at the moment. Also don't be put of by the @Xml annotations, I am using MOXy for JSON generation - this isn't an XML only thing.

{


  @XmlTransient
  private int limit, offset; // Getters for these

  @XmlTransient
  private int modelLimit; // Getters for these


  @InjectLink(
            resource = ItemsResource.class,
            method = "query",
            style = Style.ABSOLUTE,
            bindings = {@Binding(name = "offset", value="${instance.offset}"),
                @Binding(name = "limit", value="${instance.limit}")
            },
            rel = "self"
  )
  @XmlElement(name="link")
  private String href;

  @InjectLinks({
    @InjectLink(
          resource = ItemsResource.class,
          style = Style.ABSOLUTE,
          method = "query",
          condition = "${instance.offset + instance.limit < instance.modelLimit}",
          bindings = {
            @Binding(name = "offset", value = "${instance.offset + instance.limit}"),
            @Binding(name = "limit", value = "${instance.limit}")
          },
          rel = "next"
    ),
    @InjectLink(
          resource = ItemsResource.class,
          style = Style.ABSOLUTE,
          method = "query",
          condition = "${instance.offset - instance.limit >= 0}",
          bindings = {
            @Binding(name = "offset", value = "${instance.offset - instance.limit}"),
            @Binding(name = "limit", value = "${instance.limit}")
          },
          rel = "prev"
  )})
  @XmlElement(name="link")
  @XmlElementWrapper(name = "links")
  @XmlJavaTypeAdapter(Link.JaxbAdapter.class)
  List<Link> links;

  ....
}


The original porting of the declarative linking code that exists in version Jersey before 2.8 had very naive code with regards working out what the URI should be for a particular resource, it couldn't deal with any resources that were not at the root of the application, nor would it cope with query parameters which are so important when dealing with collections.

In theory there could be more than one URI for a particular resource class; but this code does need to assume a 1:1 mapping, the current implementation contains a simple algorithm that walks the Jersey meta-model to try to work out the structure, is this doesn't work in your can you can simple provide another implementation of ResourceMappingContext.

Some may question why should I use these ugly annotations when it might be easier just to inject the URI myself? Well the reason is to provide metadata that other tools can use. One of my next jobs is to extend this work to generate the hypermedia extensions and for this I need the above metadata. (Waiting on a pull request being approved before I can really get into it).

Finally it is worth noting that the paging model has its own problems which become apparent if you think of a REST collection as some kind of array that you can safely page over. Concurrent updates along with the lack of state mean that the client can never be sure they have the complete model and should expect to see some items more than once as the model is updated. Cursor or linking based schemes should be considered instead which is yet another good reminder as to why you should always treat the URI as opaque - the sever might need to changes its structure in the future. But that is a entirely different blog for another day.....

Update: See the following blog for some workarounds when reading and writing Link objects.