What actions of President Johnson drove moderate Republicans in Congress into cooperation with radical Republicans?

In general, Andrew Johnson drove the moderate Republicans
in Congress to cooperate with the radicals when he proposed Reconstruction policies that
were too lenient to the South.

For example, Johnson's plan
did not force many of the men who had led the South to step down from their positions
after the war.  He pardoned so many ex-Confederates that even moderate Republicans came
to oppose him.

In addition, Johnson did some things that
made the moderates question his sense.  In one instance, Johnson gave a speech in which
he criticized the Congressional leaders of Reconstruction very harshly and even implied
that they had thrown out the whole concept that the Union was indestructible.  In other
words, he was accusing Congress of throwing out the whole idea behind the Civil War. 
This was a very inflammatory thing to say and it helped to push moderates even farther
away from him.