Rayleigh damping is not implemented for all the elements. Since you asked for mass damping and had only a lumped mass that works fine. However, the default element type does not have the damping implemented. If you add to your material record: SHEAR OFF you will get the standard bending element using Hermite cubic interpolation for the displacements -- which has no shear deformation.
If you look at the coding in this element (frams3d.f) you should be able to modify the other elements should you need one with shear deformation.
We suggest also that you use multiple batch-end sets in your data file to help separate the information that follows the 'end'. Also, it was necessary to remove the 'tol,,1.e-02' record since no iterations were done with it in. A modified IT12 file is attached.
With the changes the damping is the nearly the same for both mass and stiffness damping.