A spokeperson for 2 Sisters confirmed that following a strategic review with M&S, the majority of employees at its Avana bakery site at Newport were now at risk of redundancy, "following the loss of our M&S business to a competitor".
"The work will now be transferred to the other company, therefore placing the majority of our 650-strong workforce at risk of redundancy. There is also a risk that the site could be closed."
The multi-million pound cake contract is believed to have been won by Park Cake Bakeries in Oldham, according to the BBC, which reported that the M&S deal makes up 85% of what the Avana bakery business produces.
The spokesperson continued: "Clearly this is a huge setback to our cakes business and our operations in Wales, but we are determined to work hard to reverse the situation.
"As a matter of urgency, we are now making every effort to rebuild the business by seeking alternative customers and opportunities which will minimise the impact of this contract loss. We are hopeful that we can retain our cakes business in Newport."
The firm will now enter into a 45-day minimum consultation period with staff to "explore all available options before taking the next steps".
Avana Bakeries, part of the Brookes Avana, was acquired by Solway Foods, part of 2 Sisters Food Group, from Premier Foods, in January 2012.
The £30m sale included three manufacturing sites in Newport (RF Brookes and Avana Bakeries) and Leicester (RF Brookes).
Following the sale, the decision was made to sell the Avana Christmas pudding business to Sargents Bakeries, following an investigation by the Office of Fair Trading (OFT) over concerns about loss of competition in the market.