A group of Sri Lankans who were trying to reach Australia when their boat became stranded off Indonesia during bad weather are set to be "escorted" back towards India, officials say.

A pregnant woman and nine children are among the 44 people on board the boat that has been drifting around 300 metres off the coast of Lhoknga in Aceh since Saturday.

Following a meeting late on Wednesday between local and central government officials, including Vice President Jusuf Kalla, authorities say those on board will be allowed to come ashore while their boat is fixed, refuelled and supplied with food.

But then they must leave.

Describing the Sri Lankans as "illegal immigrants", the head of the Aceh immigration division A. Samadan said they had been rejected on four grounds, including a lack of travel documents and a failure to notify authorities of their intended arrival.

The boat will be escorted out of Indonesian waters towards India, Mr Samadan told AAP on Thursday.

However, he could not confirm when this would happen.

Amnesty International hit out at Indonesia for not allowing those on board to be properly processed by the United Nations High Commissioner for Refugees.

The boat, flying Indian flags, began a hazardous journey from India after those on board reportedly fled Sri Lanka, where members of the Tamil minority have suffered past persecution.

The group had been at sea for 20 days in the hopes of seeking asylum in Australia before bad weather struck off Aceh.

"These people have endured a long and difficult journey already," Josef Benedict from Amnesty International said in a statement on Thursday.

"Refugees and asylum seekers frequently travel without identity documents, as often these documents are either difficult to obtain or get lost during the journey. This has no consequence on these peoples right to seek asylum. UNHCR should be allowed to register them immediately."

The comments come just days after two men aboard the boat tried to swim to shore on Monday afternoon before they were picked up and sent back to the boat by the Indonesian navy.

There are currently more than 13,000 refugees and asylum seekers registered with the UNHCR in the archipelago and detention centres in Indonesia are at over-capacity.

In Indonesia alone, the number of Afghan asylum seekers awaiting a solution has increased from about 1300 in 2010 to more than 6000 in 2015.

(AAP)