He's beloved by the gossip culture of Manhattan and was recently embraced by the left for hurling insults at CNBC "Mad Money" host Jim Cramer. But as Cramer's CNBC "Fast Money" colleagues explained, if you listened to NYU professor Nouriel Roubini, you would have missed out on a lot of stock market upside.

Roubini, often called Dr. Doom and known for crazy parties, predicted back in 2005 the speculative housing bubble would be the eventual undoing of the economy - and he was correct. However, as Jeff Macke, founder and president of Macke Asset Management and panelist on "Fast Money" explained May 11, being two years early with that prediction wasn't something to hang your hat on.

"Let me give you a little hint on trading," Macke said. "If you're two years early on any idea, what you are mostly is dead. You're a professor, as opposed to a trader. And if we still have time to talk after the five-minute butt kissing we gave the guy, I'll tell you what - he hasn't made anyone a cent. Until he does, as far as I'm concerned, it's a nice opinion but it's not making me money."

And Tim Seymour, the founder and managing partner at Seygem Asset Management, noted Roubini's recent warnings - S&P 500 to 600 (at 900 now), even though there's a sign a bottom is being formed in the S&P. He also said - if you had listened to Roubini in 2005, you would have missed out on the market's rise Dow Jones Industrial Average (DJIA) from 10,700 at the end of 2005 to a little over 14,000 in mid-October 2007.  And the S&P 500 (S&P) made a similar increase, from 1250 at the end of 2005 to 1550 mid-October 2007.

"You know, your basic trade and recession books have told anybody who has been reading on this over the last six to nine months is the markets tend to rally six to nine months even before the end of the recession," Seymour said. "Here he is a month ago telling us we're going down to 600, when in fact we are already starting to see some of these signs of at least a bottoming. He's not a trader as Jeff [Macke] pointed out and he comes into 2005 with these prognostications. We go into the highest credit market boom in history. You know, that's not a great show."

And Pete Najarian, another panelist on "Fast Money" and co-founder of optionMONSTER.com, said Roubini's doom-and-gloom predictions aren't for a cautious trader because if his advice were followed, traders would miss out on rallies.

"It's still a market made for caution," Najarian said. "Trade with caution, you protect yourself along the way. You don't have to go out there and just put everything on the line and lose it all and that's the issue sometimes I think Nouriel gets a little bit too far from."