Dic: can ill afford (to do) something

6. PHRASE : PHR to-inf, PHR n
If you say that someone can ill afford to do something, or can ill afford something, you mean that they must prevent it from happening because it would be harmful or embarrassing to them.[FORMAL]   

• It's possible he won't play but I can ill afford to lose him.

.... Cobuild

ill: Adverb
1. badly, wrongly: the title ill befits him
2. with difficulty; hardly: we can ill afford another scandal 

.... Collins Essential,

http://www.thefreedictionary.com/ill