[Originally published in The Boston Globe.]

US NATIONAL ARCHIVES

The gas crisis of the 1970s left many motorists stranded.

THE GLOBAL energy market can be a scary place for America. For decades, one of the biggest reasons has been the cartel known as OPEC.

Saudi Arabia and the 11 other nations that make up the Organization of Petroleum Exporting Countries collude openly, setting production limits and shaping the world oil market in their interests. Concerns about OPEC have driven American energy policy ever since a devastating six-month embargo by Arab oil producers in 1973 plunged the nation into recession and seared the four-letter acronym into the national consciousness.

Today the group still holds 80 percent of world oil reserves; ambassadors from the most powerful economies in the world attend its biannual meetings with deference, and dangle aid and other enticements in the hopes of winning OPEC’s allegiance. With American antagonists like Iran and Venezuela in its membership, OPEC amplifies the ability of relatively small countries to buck the desires of Washington.

But a closer look at OPEC’s real influence over the oil market suggests that we’re making a huge mistake about its global power, says Brown University political scientist Jeff Colgan. A specialist in oil and global conflict, Colgan tracked almost three decades of oil production data and compared it to official OPEC policy, which sets quotas for member countries. What he found surprised him: OPEC’s decisions were all but irrelevant.

As formidable as OPEC is seen to be, its members appeared to produce whatever they felt like, regardless of official policy; Colgan found that OPEC decisions weren’t actually affecting world oil supplies, or world oil prices. The group seemed unable to control its members or accomplish the one thing that even its detractors might appreciate: bring stability to the market.

“It drives me nuts,” Colgan says. “Washington spends bandwidth on OPEC that could be better dedicated to something else.”

Colgan’s research, published this summer, made a splash within the small circle of OPEC scholars, and even his critics concede that his findings require a reassessment of our understanding of the cartel. His thinking has yet to trigger policy changes, however. Although skepticism about OPEC has been rising—just last week, New York Times columnist Joe Nocera wrote about a 2013 Foreign Policy article titled “The End of OPEC”—most policy makers and academics still consider OPEC the key player in world energy markets, and the only one in a position to unilaterally disrupt the global flow of petrochemicals.

If Colgan is right, the implications go beyond OPEC: They suggest that petroleum is not the global bugaboo that many politicians and policy makers think. In this argument, Colgan has company: His findings echo earlier research suggesting that today’s American economy is no longer vulnerable to shocks in oil prices, or temporary supply disruptions caused by Middle Eastern wars.

His meticulous research suggests that OPEC is a sort of high-level con, which awards its member states unwarranted influence, wastes US time and energy, and distorts our energy policy and even our military priorities. An honest reckoning of power in the oil market might not only lead the United States to fear OPEC less, but even to behave a little more like it.

WHEN OPEC was formed in 1960, the oil industry was dominated by a different cartel. It was called the “Seven Sisters,” and was made up of western companies. Many of them have changed their names since then but are still industry giants, like ExxonMobil, BP, and Royal Dutch Shell.

The developing countries that actually held the world’s oil reserves wanted more clout. Saudi Arabia, which had the world’s largest and most accessible oil fields, was joined by four other founding members: Kuwait, Iraq, Iran, and Venezuela. Soon, nine more nations joined the group and opened a headquarters in 1965 in Vienna, the home of other important international institutions like the International Atomic Energy Association.

OPEC became a household name after the infamous oil embargo of 1973, which left a lasting psychological imprint on Americans. Gas stations closed on Sundays. Customers waited in interminable lines for their ration. Homeowners and businesses couldn’t afford to leave their heaters running at full blast throughout the winter. The economy went into a tailspin.

Forgotten in the bitter memory is that the embargo wasn’t actually imposed by OPEC, but by the Arab members of the cartel, along with Egypt, Syria, and Tunisia, in retaliation for America’s support for Israel in the October 1973 Arab-Israeli War. That distinction was lost, and policy makers ever since have railed against the dangers of dependence on OPEC oil. The legacy of the oil embargo drives American diplomacy, the rules governing worldwide oil contracts, and even the US case for hydraulic fracturing, or fracking, which contends that the political benefits of “energy independence” outweigh fracking’s environmental and economic drawbacks.

DAVID BUTLER/GLOBE STAFF

Today, economists point out, the world energy market is far more integrated and interdependent than it was in 1973, when most oil was bought and sold in bulky, long-term contracts that made it hard for the market to quickly adjust to any change in supply.

Now producers need the profits as much as consumers need the gas. And despite the size of OPEC’s reserves—half of which are held by just two countries, Saudi Arabia and Venezuela—oil production is far more widely spread out than it used to be. Countries like the United States, Canada, and Mexico can satisfy a great deal of short-term demand even if their supplies will run low in a few decades. (In fact, the recent surge in US oil production last year made it the world’s largest oil producer, though its reserves are limited and the extraction process is only profitable when oil prices are high.) Oil is now bought and sold in a market that changes daily, so if one supply suddenly goes offline—like the oil industries of Libya and Iraq during various points of the last decade’s turmoil—other countries can step in to fill the gap in a matter of days.

Political scientists and economists have explored OPEC’s efficacy in multiple papers over the years, and almost all of them have concluded that even if it doesn’t function as a seamless cartel, it is the single most pivotal factor in setting global oil prices. It is this consensus that Colgan’s research punctures. He looked at official quotas since 1982, and found that OPEC member countries cheat an astonishing 96 percent of the time, pumping more than their permitted quota. He created a mathematical model to predict how much oil each country would produce if it were not constrained by the cartel’s quotas, and he found that when it came to a country’s oil production patterns, it didn’t seem to matter whether it was in OPEC or not. New members didn’t reduce production when they joined OPEC, and quota changes didn’t affect production levels.

Despite its reputation, Colgan found, OPEC simply doesn’t fit the definition of an effective cartel. Saudi Arabia—the sole producer with the spare capacity huge enough to unilaterally alter world supplies—floods the market or slashes capacity to suit its own needs, as it did in 2008 and is threatening to do again today in order to drive US fracking companies out of business. Almost all of the time, other OPEC members pumped as much as they could, whether prices were high or low.

Michael Levi, an energy and oil expert at the Council on Foreign Relations, acknowledges Colgan’s point that OPEC’s control of production and prices is not absolute, but believes he’s going too far in calling it powerless; cartels by definition aren’t transparent, and OPEC might still wield plenty of influence over member behavior. “It would be awfully unwise for policy makers or market participants to quickly flip to an equally over-confident belief that OPEC doesn’t matter,” he says.

American politics pretty much guarantees they won’t flip soon: In today’s debate over whether the United States should export its own oil, it’s still OPEC whose wrath the White House fears, rather than the more likely retaliation it might face from individual countries like Saudi Arabia. And OPEC is a convenient punching bag on Capitol Hill: Since 1999, the US Congress has introduced no fewer than 15 versions of a “NOPEC” bill, which would require the government to punish members of the international oil cartel. All the bills have failed, but they attract high-profile support. When they were senators, both Barack Obama and Hillary Rodham Clinton voted for NOPEC bills.

COLGAN CALLS the cartel’s reputation a “rational myth”—a made-up story perpetuated because it serves an interest. OPEC initially was founded to control the oil market, but by the time member countries realized it didn’t, they were reaping too many political benefits from OPEC’s perceived clout to dissolve the organization.

DAVID BUTLER/GLOBE STAFF

OPEC membership has unquestionable benefits on the world stage: Colgan measured the number of ambassadors to members and found that joining OPEC provides a noticeable bump in foreign missions. When countries like the United States are worried about global oil production levels, or prices, they make pleas to the biggest player in the market, and that means OPEC.

For America, though, the fear of OPEC has costs. For one thing, it means the United States misses opportunities to exploit the fissures between OPEC countries, which often have diametrically opposed interests (today for instance, Iran wants low production and high prices to help it survive sanctions; Saudi Arabia, meanwhile, wants low prices in order to regain its dominant market share). Since the 1973 embargo, almost every aspect of US energy policy appears designed to protect consumers and the economy from a price shock or supply disruption, even though today the United States is itself an oil giant that gets rich off the sale of oil and gas.

There are real lessons to take from OPEC as we have long understood it—and from comparing countries that have wisely managed their oil wealth, like Norway, to those that have used it to mask domestic stagnation, like Saudi Arabia and Venezuela. Whether a country is an oil exporter or importer, it’s a smart investment to reduce consumption and diversify sources as much as possible, including toward wind and solar power. The most impressive oil exporters husband their energy profits, treating them as a limited windfall rather than a sustainable and permanent revenue stream.

The experience of the OPEC countries also highlights the tension between gas pricing, environmental stewardship, and national interests in ways that are increasingly relevant for the United States. Traditionally, low fuel prices have boosted the US economy, but increased pollution and dependency. High gas prices are good for an energy policy built around restraint—less consumption, less pollution—and now they actually have an economic benefit as well, boosting the burgeoning domestic oil sector.

Even if OPEC is not the power we thought, the group’s recent history has lessons for us, most simply that it’s not a bad idea to maximize the profits you can draw from your limited reserves of underground oil. Pump less to drive prices up, pump more when you need cash (or extra energy), and worry less about the global economy than about your own bottom line and long-term fiscal health. That might be the formula of a villainous cartel—or just good business sense for a nation.

[Originally published in The Boston Globe Ideas section.]

EVER SINCE AMERICANS had to briefly ration gas in 1973, “energy independence” has been one of the long-range goals of US policy. Presidents since Richard Nixon have promised that America would someday wean itself of its reliance on foreign oil and gas, which leaves us vulnerable to the outside world in a way that was seen as a gaping hole in America’s national security. It also handcuffs our foreign policy, entangling America in unstable petroleum-producing regions like the Middle East and West Africa.

Given the United States’ huge appetite for ­fuel, energy independence has always seemed more of a dream than a realistic prospect. But today, nearly four decades later, energy independence is starting to loom in sight. Sustained high oil prices have made it economically viable to exploit harder-to-reach deposits. Techniques pioneered over the last decade, with US government support, have made it possible to extract shale oil more efficiently. It helps, too, that Americans have kept reducing their petrochemical consumption, a trend driven as much by high prices as by official policy. Total oil consumption peaked at 20.7 million barrels per day in 2004. By 2010, the most recent year tracked in the CIA Factbook, consumption had fallen by nearly a tenth.

Last year, the United States imported only 40 percent of the oil it consumed, down from 60 percent in 2005. And by next year, according to the US Energy Information Administration, the United States will need to import only 30 percent of its oil. That’s been driven by an almost overnight jump in domestic oil production, which had remained static at about 5 million barrels per day for years, but is at 7 million now and will be at 8.5 million by the end of 2014. If these trends continue, the United States will be able to supply all its own energy needs by 2030 and be able to export oil by 2035. In fact, according to the government’s latest projections, the country is on track to become the world’s largest oil producer in less than a decade.

Yet as this once unimaginable prospect becomes a realistic possibility, it’s far from clear that it will solve all the problems it was supposed to. As much as boosters hope otherwise, energy independence isn’t likely to free America from its foreign policy entanglements. And at worst, say some skeptics who specialize in energy markets, it might create a whole new host of them, subjecting America to the same economic buffeting that plagues most oil exporters, and handing China even more global influence as the world’s behemoth consumer.

As much as the shift brings opportunities, however, it is also likely to open the United States up to liabilities we have not yet had to face. The consequences may be both good and bad, enriching and destabilizing for US interests—but they will certainly have a major impact on our geopolitics, in ways that the policy world is only just beginning to understand.

***

WHEN RICHARD NIXON was president, America consumed about one-third of the world’s oil, importing about 8.4 million barrels per day chiefly from the Middle East. The status quo hummed along until the Arab-Israeli war of 1973. The United States sent weapons to Israel, and the Arab states retaliated with a six-month oil embargo, refusing to sell oil to America. It was the only time in history that the “oil weapon” was effectively used, and it made a permanent impression on the United States.

Over time, the American response to the embargo came to include three major initiatives that still shape energy policy today. First, the government promoted lower oil consumption by pushing coal and natural gas power plants, home insulation, and mileage standards for cars. Second, the country drilled for more of its own oil. Third, and perhaps most important from a foreign-policy standpoint, the United States promoted a unified global oil market in which any country had the practical means to buy oil from any other. That meant that even if some countries couldn’t do business with each other—say, Iran and the United States—it wouldn’t affect the overall price and availability of oil. Other countries could fill in the gap.

The dreams of energy independence crossed party lines. Though liberals and conservatives differ on the means—how much we should rely on new drilling versus energy conservation—both parties have endorsed the quest. It was one of the few issues on which Presidents Carter and Reagan agreed.

America has made steady progress over the years, to the point where the nation’s total oil consumption has actually begun to drop. As this has happened, the high cost of global energy has also made it profitable to increase domestic production of natural gas and oil. A few months ago, both the US Energy Information Administration and the International Energy Agency predicted that if current production trends continue, the United States will overtake Saudi Arabia and Russia as the world’s largest oil producer in 2017.

Taken together, our slowing appetite and booming production mean that with a suddenness that has surprised many observers, the prospect of energy independence—technically speaking, at least—looms in the windshield.

Energy independence looks different today, however, than it did in the oil-shocked 1970s. For one thing, the energy market is a linchpin of the world order, and any big shift is likely to have costs to stability. Some analysts have warned that America’s growing oil production will create a glut that lowers prices, eating up the profits of oil countries and destabilizing their regimes. (That’s in the short term, anyway; worldwide, oil demand is still rising fast.) Falling prices mean that countries that depend on oil will face sudden cash shortages. It’s easy to imagine how destabilizing that could be for a natural-resource power like Russia, for the monarchs of the Persian Gulf, or for the dictators in Central Asia. No matter how distasteful their rule, the prospect of an unruly transition, or worse still, a protracted conflict, in any of those countries could cause havoc.

In the long term, this is not necessarily a bad thing: Weakening oppressive or corrupt governments could ultimately be beneficial for the people of those countries. And a shift in the balance of power away from the Gulf monarchies of OPEC and toward the United States could have a democratizing effect. In any event, though, lower oil prices and a dynamic energy market make the current stable order unpredictable.

China’s economic rise has also changed the global energy equation. For now, China is largely without its own petroleum supplies and is ­replacing the United States as the largest importer. As China steps into the United States’ shoes as the world’s largest oil customer, it will gain influence in oil-producing regions as American influence wanes. It might also feel compelled to invest more heavily in an aggressive navy, fearing that the United States will no longer shoulder the responsibility of policing shipping lanes in the Persian Gulf and elsewhere—a costly security service that America pays for but which benefits the entire network of global trade.

Domestically, there’s also the “resource curse,” which afflicts countries that depend too heavily on extracted commodities like minerals or petroleum. Such industries don’t add much value to a society beyond the price the commodity fetches at market, and that price is notoriously fickle, meaning fortunes and jobs rise and fall with swings in global prices. The resource curse often implies corruption and autocracy as well. But economists are less concerned about that, since the United States already has an effective government and laws to thwart corruption, and because oil will still make up a minuscule overall share of the economy. Last year oil and gas extraction amounted to just 1.2 percent of the American gross domestic product.

***

THERE ARE STILL plenty of people who think that energy self-sufficiency will be an unalloyed good. Jay Hakes, who has pursued the goal as an energy official under the last three Democratic presidents, says that America will reap countless political and economic dividends. It will help the trade deficit, give American companies and workers benefits when oil prices are high, and insulate the country from supply shocks. It will also give Washington wider latitude when dealing with oil-producing countries, on which it will depend less. “There are some downsides,” he acknowledges, “but they’re outweighed by all the positives.”

One benefit that self-sufficiency won’t bring, it seems clear, is a sudden independence from the politics of the Middle East. The region produces about half the world’s oil, and Saudi Arabia alone has so much oil that it can raise its capacity at a moment’s notice to make up for a shortfall anywhere else in the world.

Already, America is largely independent of Middle Eastern oil as a consumer: Only about 15 percent of our supply comes from the region. But we do depend on a stable world market—even more so if we become a net exporter ourselves. So even if we don’t buy Saudi oil, we’ll still need a stable Saudi regime that can add a few million barrels a day to world flows, at a moment’s notice, to offset a disruption somewhere else.

Michael Levi, a fellow at the Council on Foreign Relations and author of the book “The Power Surge: Energy, Opportunity, and the Battle for America’s Future,” believes that the biggest risk of achieving a goal like energy independence is complacency: Without the pressures that importing oil has brought, we may have little reason to innovate our way out of fossil fuels altogether. The policies themselves have achieved a great deal of good, he points out—stabilizing the world’s energy markets, reducing consumption, and pushing us beyond “independence” toward renewable sources like wind and solar power (though today these still make up a vanishingly small portion of the US energy supply).

Levi argues that an American oil bonanza could easily remove the political incentives for long-term planning and sacrifice. “I get scared that we’ll become complacent and make foolish decisions because we believe we’ve become energy independent,” Levi says. Energy independence was a useful slogan to motivate America, but in reality, a sensible energy policy has to balance a plethora of competing concerns, from geopolitics and the environment to consumer demand and fuel’s importance to the economy.

“The real way to be energy independent,” he said, “is actually to not use oil.”

John Levi Barnard has written a powerful appeal to for us to take moral heed of the oil geyser in the Gulf of Mexico, abandon our wasteful ways, and embrace a kinder new paradigm.

There is a lesson to be learned here, as there was from the crisis of the 1850s, which is that there is a clear choice to be made between principle and the expedient solution, which is never a solution at all. Every president from Reagan to Obama has assured us that the American way of life is not negotiable, much as the great compromisers of the antebellum period assured us that nothing, not even Justice, would threaten the cohesion of the “union.”

Thoreau countered this in no uncertain terms, declaring in “Resistance to Civil Government” that the American “people must cease to hold slaves, and to make war on Mexico, though it cost them their existence as a people.” From the crisis in the Gulf of Mexico—which extends both literally and figuratively to the Persian Gulf and the borderlands of Pakistan—we can derive a similarly radical statement: we must stop drilling in the ocean though it costs us something at the pump, though it forces us to make our way of life negotiable.