In the process, my reporting has found, common crawl has opened a back door for ai companies to train their models with paywalled articles from major news websites. Nonprofit common crawl, best known for making open web archives that are used to train ai models, is under fire for claims that its troves have allowed ai companies to ingest paywalled journalism at scale. According to the report, common crawl, whose database spans multiple petabytes, has effectively opened a backdoor that allows ai companies to train their models on paywalled content from.
Mountain mama | Scrolller
Common crawl maintains archives containing millions of articles from major news organizations that readers typically must pay to access, enabling ai developers including openai, google, anthropic, nvidia, meta, and amazon to train their models on premium journalism without compensation to publishers.
![[WV] Mountain mama : SkyPorn](https://i2.wp.com/preview.redd.it/wv-mountain-mama-v0-lg8wz7rlozv91.jpg?auto=webp&s=57f3e04dbd700a0de099bb0d781845684864a462)
