Pacific youth growing up in rural horticultural towns in Victoria, Australia, frequently experience the stigmatising labels of ‘fruit picker’ and ‘farmworker’. Drawing on extensive fieldwork in the Sunraysia region, this paper focuses on schools as early sites of racialisation and racism, and compare the perspectives of students and teachers. While teachers often blame ‘culture’ for poor academic performance, students complain of teachers’ racism while also experiencing low confidence and poor self-esteem that hinder their academic progress. Underpinning these issues is the racial hierarchy in the towns, shaped by their particular histories of colonisation, migration and intercultural relationships. Current theories of rural multiculturalism tend to focus on everyday interactions and strategies to encourage social cohesion, with the inherent danger of neglecting the deeper structural issues. The authors that Pacific youth seeking to move beyond the casual and precarious farm work undertaken by their migrant parents encounter structural barriers that begin in schools and profoundly shape their future opportunities.