Turkey may find it hard to play a public role in the coalition the US is building to strike at Islamic State targets in Iraq and possibly Syria for fear the militant group might retaliate against dozens of Turks held hostage.

President Barack Obama has said he hopes to devise a regional strategy to try to counter IS, which has seized swathes of territory in Iraq and Syria, but current and former US officials say they expect Turkey to avoid any major public role.

An ally in the US-led Nato military alliance, Turkey is the only Muslim nation in a "core coalition" of 10 countries committed to battle IS militants in Iraq that the US announced on Friday at the Nato summit in Newport, Wales.

It is unclear to what extent the coalition may target IS in Syria, where the Islamist militants enjoy safe haven.

The bulwark of Nato's southeastern flank, Turkey has sensitive relations with seven sometimes unstable neighbours, bordering not only Syria, the origin of the IS threat, but Iraq, where the Islamist group took dozens of Turks hostage.

IS is holding some 46 Turkish citizens hostage, including diplomats seized from the Turkish consulate in Mosul when it overran Iraq's second-largest city in June.

In deference to Turkey's predicament, Washington aims to have Ankara focus on halting the flow of foreign militants, including many from the United States and Western Europe, who have crossed its territory to join the fight in Syria.

"Everybody understands that the Turks are in a special category," said a US official on condition of anonymity, alluding to the safety of Turkey's hostages and the reluctance of one neighbor to attack another for fear of retaliation.

"Turkey will be part of the coalition but what does that mean? It doesn't cost much to get your flag up on the wall."

A second official said Washington would ask Ankara to do more to keep foreign fighters out of Syria, a message Obama delivered diplomatically on Friday and Defense Secretary Chuck Hagel is expected to echo in Turkey next week.

When he met Turkish President Tayyip Erdogan in Newport, Obama told reporters: "I want to express my appreciation for the cooperation between US and Turkish both military and intelligence services in dealing with the issue of foreign fighters, an area where we still have more work to do."

Turkey has long sought to spur the United States to greater action to end the 3-1/2-year-old Syrian civil war in which an estimated 200,000 people have died.

Turkey has drawn up a "no-entry" list of 6,000 people suspected of seeking to join "extremists in Syria" based on information from foreign intelligence agencies, Turkish officials have said.

"The foreign fighter issue is well-known. It's a problem. It's certainly fueling ... their advance in Syria and in Iraq," said the second US official. "That would certainly be something that we would look for help from Turkey on."

The Islamic State's lightning advance across Iraq led Obama to resume air strikes in the country in August for the first time since he withdrew US troops in 2011. As of Thursday, the United States had carried out 127 such strikes.

Expectations have grown that he may target IS in Syria. In addition to sending Hagel to Turkey he is dispatching Secretary of State John Kerry to the Middle East next week to corral coalition partners.

Turkey's dilemma illustrates the sort of challenges that Kerry faces pulling together an active coalition among states with very different interests and constraints in the region.

Reflecting the sensitivity of the issue in Turkey, Turkish officials have privately stressed that none of the US aircraft attacking Iraq came from Incirlik Air Base in southern Turkey.

"They will not allow the use of Incirlik for lethal air strikes," said Henri Barkey, a Lehigh University professor and former member of the State Department policy planning staff.

"The Foley execution was a message not just to us, but also to the Turks," he added, referring to an IS video released on August 19 that showed a masked fighter beheading US journalist James Foley. A video showing the execution of a second US journalist, Steven Sotloff, became public on Tuesday.

While stressing their commitment to the North Atlantic Treaty Organization, which Turkey joined in 1952, Turkish officials voiced an unwillingness to jeopardize their hostages.

"We are a Nato ally. We share the same principles and values with the West. But we have our 46 hostages," said one Turkish official who spoke on condition of anonymity.

A Western diplomat in Ankara said: "They are hamstrung. They can't, understandably really, sign up with enthusiasm to military action. I'm sure privately they are supportive but they ... don't feel they're able to come down definitively in public."

US-based analysts, including retired Lieutenant General David Barno, a former top US commander in Afghanistan, said Turkey could help behind the scenes by allowing tankers, reconnaissance planes and drones to operate from its territory.

"It's a higher probability they might be willing to use those bases for non-lethal activities ... as opposed to aircraft carrying bombs," said Barno, who is now at the Center for a New American Security think tank in Washington.

To go after IS targets in Syria, however, countries such as Turkey, Saudi Arabia, Jordan and others may need to be convinced of US staying power despite Obama's reluctance to get involved in another Middle East war.

"Anybody who is going to give serious thought, and ultimately agree to join a coalition, must be utterly convinced by the president of the United States that the United States is there for the duration, that nobody is going to be left hanging out to dry," said Fred Hof, a former State Department Syria expert.-Reuters