Dic: fall away

1.  Also, fall off. Withdraw one's friendship, support, or allegiance. 

• After the divorce, her friends slowly fell away. [Early 1500s] 
  

2.  Also, fall off. Gradually decline in size or strength. 

• The breeze slowly fell away. 
• "Love cools, friendship falls off, Brothers divide." Shakespeare(King Lear, 1:2)
  [Early 1500s] 
  

3.  Drift from an established faith, cause, or principles. 

• I fell away from the Catholic Church when I was a teenager. [Early 1500]
  

Am Heritage Dic of Idioms, http://www.answers.com/topic/fall-away