Let 
 and 
 be an Eisenstein series and a cusp form, respectively, of the same weight 
 and of the same level 
, both eigenfunctions of the Hecke operators, and both normalized so that 
. The main result we prove is that when 
 and 
 are congruent mod a prime 
 (which we take to be a prime of 
 lying over a rational prime 
), the algebraic parts of the special values 
 and 
 satisfy congruences mod the same prime. More explicitly, we prove that, under certain conditions, \[ \frac{\tau (\bar{\chi })L(f,\chi ,j)}{(2 \pi i)^{j-1}\Omega _f^{\text{sgn}(E)}} \equiv  \frac{\tau (\bar{\chi })L(E,\chi ,j)}{(2 \pi i)^{j}\Omega _E} \pmod{\mathfrak{p}} \] where the sign of 
 is 
 depending on 
, and 
 is the corresponding canonical period for 
. Also, 
 is a primitive Dirichlet character of conductor 
, 
 is a Gauss sum, and 
 is an integer with 
 such that 
. Finally, 
 is a 
-adic unit which is independent of 
 and 
. This is a generalization of earlier results of Stevens and Vatsal for weight 
. The main point is the construction of a modular symbol associated to an Eisenstein series.