The Swedish hacker is serving 50 weeks for violating his bail conditions after seven years holed up at the Ecuadorian Embassy in London.
Assange, 47, faces extradition to the US over his Wikileaks activities.
He met Nils Melzer who usually reports on prison torture in corrupt states at HMP Belmarsh, South East London on May 10.
He was also visited by human rights expert and privacy campaigner Joseph Cannataci on April 25.
The source said: "The officials refused to follow normal security procedures to have an officer with them.
'A POLITICAL PRISONER'
"Assange barely stopped talking. He thinks he's a political prisoner."
It is understood jail staff initially insisted on having a guard in the room. But the UN pair told him to stand outside so he could not listen in. They were set to see Assange at the Ecuadorian embassy but his arrest on April 11 ended that plan.
Mr Melzer said after his meeting: "I will be trying to assess the risks of torture and ill treatment Mr Assange might be exposed to."
A spokesman for Melzer who has also reported on alleged prisoner maltreatment at Guantanamo Bay's Camp X Ray said: "Just three people were present, Mr Assange, Mr Melzer and an independent medical ex- pert. No guard was within earshot."
Tory MP Sir Henry Bellingham wants to know why a jail officer was not allowed to monitor the meetings.