About a year and a half ago I built myself a tool I call “translation workbench” to make reading manga (in its original Japanese) easier. Going from textbook Japanese to colloquial Japanese is a hell of a jump and there’s really nothing quite like manga for absorbing common speech.
The core problem though is, unless you can read all 2,136 常用漢字, you’re going to spending 90% of your time in the dictionary. That is assuming you have no problem with the grammar.
So I did what I do. I built a tool! Take a picture of a page, upload it into translation workbench, splash some ML+computer vision on it to detect the page
- apply keystone correction, run it through a preliminary translation engine, ENHANCE, select regions of interest + apply layout. Voila, you now have all the extracted kanji in a place where in-browser tools like rikaikun can drastically speed up reading time.
Anyway, this tool has been running on AWS and I even did the basics of productizing it so other people could use it… but then I basically got pulled in another direction. Is this something you’d like to use? Or even work on? The market of English-speaking Japanese learners that get far enough to attempt manga is relatively small, but I keep thinking about it anyway. I mean, I built all this, so maybe it’s worth something to someone other than me.
Who doesn’t love screenshots?
Pages of a book so you can see where they are in the workflow
Corner detection + the raw image. The red square shows where it thinks the upper left corner of the page is.
Regions of interest. Green is what the OCR system found, blue is what I want extracted.
Final stage of workflow where you can hover for single word translation.
