Majority of workers concerned fiscal cliff will lead to layoffs