Groups representing landowners claimed they were "frightened" by agents turning up on their farms.