Their debut album Kyouiku is more definitely more rocky, so you might want to try that out first. I like how the opening track is a rock version of a Shiina Ringo track, which sets the stage well and sends a nice message, about her departure from her solo work, and what to expect from Tokyo Jihen.
However, I personally like their next album, ADULT, the best. I am not actually a very rock person, and ADULT is definitely more jazz-infused. I did not like their third (VARIETY), and while the fourth album (SPORTS) is a step up from the third. I would rank them the following in terms of my fav:
However their new album is quite promising for me. I have liked everything they released since SPORTS....