Here is some advice about sleeping for free in Japan, based on my own experience.
From July 2014 to November 2014 I travelled a bit more than one month by foot and hitchhiking in Hokkaido and then during two months and a half by bicycle, from Hokkaido to Shikoku. And excepted at the beginning in Hokkaido, I never paid for my nights !
Before I say anything else, keep in mind that Japan is very safe and that your problem is mainly to avoid policemen.
The best plan to sleep for free is to go in a michi no eki. It litteraly means "station of road". It is a place where there are toilets opened all the time. Very often there are maps, a restaurant and a shop with local food (vegetables, specialities of the place...) and free wifi.
To find them you need to look at these kanjis : 道の駅.
There are more than a thousand of michi no eki in Japan ! Enjoy !
The very best is a michi no eki with a small parc in which you can put your tent. If there is no parc, you still can put your tent, but be carefull of the wind of course !
You won't have any trouble because even Japanese people use michi no eki to sleep. Very often it is in their car, but sometimes bikers also do so, especially in Hokkaido during summer. At that time the problem is more to get a good place before someone else take it.
And if there is no michi no eki ?? No problem, you just need to find a parc. In Japanese you have to ask for a kouen (say kooen), in kanji : 公園.
In this case it is a bit more complicated because in some famous parc you'll be woken up by the police and they'll ask you to go somewhere else (don't worry, no fine).
Avoid the parcs linked to a castle, you won't be able to sleep in them since policemen patrol every night. It happened to me in Hirosaki for instance.
If you do so, then it should be okay ! In almost every village there is parc, even if it is a smal one, and it is possible to sleep in it. Of course wait until it is dark enough to be discretfull. I did it many many times and had no problem.
About Ueno parc in Tokyo.
Since this year it is now forbidden to put a tent in this parc. I did it in July 2014 with two friends and it was alright, but when I wanted to do it again in November 2014 the policemen said no tent. It was midnight, so maybe you can do it after this time but I don't recommend to try.
Instead it is better to go to the Sumida parc, in Asakusa, along the Sumida river. There is no real good place to put a tent, but many homeless people (totally inoffensives) are living there and I think that you won't be troubled by policemen there.
If you go even more east, alongside Arakawa near Ojima Komatsugawa kouen, it's even better ! A bit farther from the center though.
Good luck and bon voyage !!