On Tuesday Mr. Holder testified in front of the House Judiciary Committee about the IRS scandal and other controversies.

As usual, the Attorney General said little citing ongoing investigations that for years have yielded no public results. In fact, Mr. Holder was cited for contempt of Congress in 2012 for failure to hand over documents about the "Fast and Furious" gun scandal. As usual, nothing happened to him. But that contempt action was cited at the hearing.

(BEGIN VIDEO CLIP)

UNIDENTIFIED MALE: I realize that contempt is not a big deal to our Attorney General but it is important that we have proper oversight.

HOLDER: You don't want to go there, buddy. You don't want to go there ok.

UNIDENTIFIED MALE: I don't want to go there?

HOLDER: No.

UNIDENTIFIED MALE: About the contempt?

HOLDER: You should not assume that that is not a big deal to me. I think that it was inappropriate. I think it was unjust. But never think that was not a big deal to me. Don't ever think that.

(END VIDEO CLIP)

O'REILLY: Now, yesterday, speaking before the National Action Network, Sharpton's advocacy group, the Attorney General ramped up the controversy.

(BEGIN VIDEO CLIP)

HOLDER: I am pleased to note that the last five years have been defined by significant strides and by lasting reforms. Even in the face, even in the face of unprecedented, unwarranted, ugly --

UNIDENTIFIED FEMALE: Yes.

HOLDER: -- and divisive adversity. You look at the way the Attorney General of the United States was treated yesterday.

UNIDENTIFIED FEMALE: Yes.

HOLDER: By a House Committee, it had nothing to do with me. Forget that. What Attorney General has ever had to deal with that kind of treatment?

UNIDENTIFIED MALE: That's right.

HOLDER: What President has ever had to deal with that kind of treatment?

(END VIDEO CLIP)

O'REILLY: Mr. Holder heavily implying the House Judiciary Committee is giving him a hard time because of his skin color.

Today on "Fox & Friends" I said this.

(BEGIN VIDEO CLIP)

O'REILLY: It's another grievance situation now. I'm not going to put words in the Attorney General's mouth that he is saying that it's because is he black. But, certainly where the venue is and he is speaking to primarily African-Americans you could take that inference.

But it's so hollow. I mean come on. You didn't find out what happened in Benghazi. You didn't find out what happened with "Fast and Furious". You didn't find out what happened with the IRS. Years go by. He should have been fired a long time ago. And I don't know what -- I don't care what color he is.

Do you think that the House committee called him in and say let's get the black guy today? Is that what they did? Does anybody believe that? Let's get the black guy today.

(END VIDEO CLIP)

O'REILLY: So, the heavy evidence is that Eric Holder is using his skin color as a shield to avoid explaining to the American people why he has been so ineffective on a variety of fronts. Or am I wrong?

It's long past time to stop this kind of nonsense. Richard Nixon was treated far more harshly by Congress than Barack Obama. And his attorney general, John Mitchell was ruined over the Watergate investigation.

The truth is that Eric Holder has been treated gently by the national press and has not been subject to much scrutiny by the public. In fact, most Americans don't even know who he is. Eric Holder's first duty is to the American people, not his party and not the President.

But we get nothing from him. He doesn't update investigations in specific ways. He holds back documents from Congress. And he stonewalls time and again. That's the truth and it has nothing to do with skin color.

And that's "The Memo."

- You can catch Bill O'Reilly's "Talking Points Memo" weeknights at 8 and 11 p.m. ET on the Fox News Channel and any time on foxnews.com/oreilly. Send your comments to: oreilly@foxnews.com.