A lack of oversight from NHS England means that patients can not be "assured" they will receive an acceptable GP out-of-hours services, the National Audit Office (NAO) claims.
NHS England commissions about 10% of out-of-hours services - ones provided directly by GP practices.
However, NHS England is doing "very little" to manage and oversee the services, an NAO report claims.
And current quality assurance arrangements are unlikely to provide meaningful assurance, as clinical commissioning groups (CCGs) are only required to answer a few yes-no questions.
But the remainder of out-of-hours services, which have been commissioned by CCGs have received positive responses from patients.
Amyas Morse, head of the National Audit Office said: "NHS England has much to do to help secure improvements throughout the system and to increase its oversight of the out-of-hours GP services it commissions directly.
"It should also work to raise public awareness of how and when patients should contact out-of-hours GP services, and needs to be prepared to take the lead in integrating these services effectively with other parts of the urgent care system.”
Out-of-hours GPs provide urgent primary care when GP surgeries are typically closed. They form part of the urgent care system, along with other services including NHS 111 and A&E departments.
The number of cases handled by out-of-hours GP services has fallen significantly, from an estimated 8.6 million in 2007/08 to 5.8 million in 2013-14. This is partly because of the introduction of the NHS 111 telephone service.
The NAO estimates that out-of-hours GP services also cost less now, in real terms, than they did in 2005/06.
The full report is available to view on the NAO website.