Supporters say that the son of an immigrant from Russia, who went from being the lone Jewish kid in Russel, Kan., to the halls of power in Washington, relished the role of underdog, even as he came to epitomize the consummate insider.
"Specter is the ultimate fighter and the ultimate winner," said Joseph Smukler, a Philadelphia lawyer and philanthropist who has known and supported Specter for more than 40 years.
Smukler was referring not only to campaign battles, but to his triumph over a brain tumor and Hodgkin's disease.
But his winning streak came to a halt on May 18, when U.S. Rep. Joe Sestak triumphed in the Democratic primary.
Now that his career in politics appears to have come to an end, many in the Jewish community are mourning the loss and hailing his decades of public service, during which he was often seen as a "go-to" guy for the Jewish community on Israel and a host of other issues.
"Anything I ever asked Arlen Specter to do with respect to Israel or Jews, he has always done it," said Morton Klein, the longtime head of the Zionist Organization of America, who stuck by Specter through thick and thin, even when he disagreed with the senator over his efforts to reach out to Iran and Syria.
Many expressed admiration for his attention to other Jewish causes as well, from advocating for Soviet Jewry to securing funding for Holocaust awareness and restitution to survivors.
Examples of Specter defying political odds are well-known: eking out narrow victories against activist Lynn Yeakel in 1992 and against Pat Toomey in 2004.
He also threw his hat into the presidential ring in 1995, confident that the country would turn toward a pro-choice Republican who railed against the religious right. Part of his platform was moving the U.S. Embassy in Israel from Tel Aviv to Jerusalem. The campaign floundered quickly.
"My father came to this country in 1911 for religious freedom," Specter said in a news conference with members of the Jewish media at the time. The longtime member of Congregation Mikveh Israel in Center City, who later joined Har Zion Temple in Penn Valley to be with his grandchildren, often invoked his father's journey from tsarist Russia to the American heartland.
"I'm just not willing to stand by and see this fringe take over our party," he said.
He did complain to a reporter that the Jewish community did not provide as much support as he would have expected.
But throughout his career, and especially in close races, Jewish support often seemed to prove the difference between victory and defeat. That happened in 1992 and again in 2004, when Specter beat Toomey by nearly 14,000 votes in heavily Jewish Montgomery County. That race was decided by less than 17,000 votes.
But the biggest gambit of all for the longtime Republican -- abandoning his party of five decades to run as a Democrat to try to secure a sixth term -- has failed. He fell victim to anti-incumbency fervor, the distrust of Democrats of the longtime Republican, a tough-as-nails challenger, and the sentiment among some that it was time for a change.
In the larger political sphere, the discussion of his legacy could last for years to come. Should he be remembered as the lawmaker who put principle before party, pragmatism over ideology? After all, he's steered untold millions in federal dollars to his home state, and been an ardent advocate for stem-cell and cancer research.
Or will history portray Specter as a lawmaker who shifted with the political winds -- who sided with President George W. Bush when it was in his interests and then quickly, President Barack Obama?
Ultimately, the lawmaker who helped torpedo the Supreme Court nomination of conservative Robert Bork in 1987 and ensure the success of Clarence Thomas in 1991 might best be remembered as a contradiction, an enigma.
Even those who disagreed with him express admiration for his service.
Among them is Betsy Sheerr, a communications specialist and political donor who promotes a pro-Israel foreign policy and a liberal domestic agenda.
Sheerr -- who sits on the board of the Jewish Publishing Group, which oversees the Jewish Exponent -- recalled first meeting Specter in 1992. Despite her objections to his role in backing Thomas, she grew to appreciate Specter's ability to get funding for cancer research, his maintaining a pro-choice stance in the increasingly pro-life centered GOP and his commitment to Israel's security.
Over the years, she not only donated to his campaigns, but briefly switched party registrations to back him against more conservative Republican challengers. But after his party switch, which shocked the political world last spring, Sheerr said her support shifted to Sestak.
"I'm not sure what compass guides him anymore, and that really disturbed me," she said of Specter.
But Klein's commitment was steadfast. Their paths also first crossed back in the 1992 Senate race when Klein, then an economist and a nascent Israel activist, raised questions about Yeakel's association with the Bryn Mawr Presbyterian, which hosted a series of events that he and others deemed anti-Israel.
Klein has publicly disagreed with Specter more than once, including over the senator's meetings with former Syrian president Hafez Assad and with his son, current president Bashar Assad, as well as his attempts to arrange a congressional delegation to Iran.
But Klein said that Specter has done much for Israel behind the scenes, including fighting to maintain U.S. military assistance to the Jewish state.