Japanese Foreign Minister Fumio Kishida talked with his Russian counterpart, Sergei Lavrov, on the phone for about an hour on Tuesday.
Japan is stepping up diplomatic efforts to help peacefully resolve the tension over Ukraine's Crimea region.
The Autonomous Republic of Crimea plans to hold a referendum next Sunday to ask residents if they want to split from Ukraine and if they want to join Russia.
Kishida said Ukraine's sovereignty and territorial integrity should be maintained, and changing the status quo by force is unacceptable.
He also said Japan is concerned about the referendum.
Kishida urged Russia to hold direct talks with Ukraine's interim government and agree to international monitors on the Crimean Peninsula.
Lavrov replied that the interim government lacks legitimacy.
Kishida later told reporters that Japan recognizes the interim government as the one representing Ukraine.
They reaffirmed that they will maintain close communication.