Three teams of data scientists worked diligently to try and mitigate the risk of sharing large data sets at the recent ACS Data Sharing Directed Ideation.

Their efforts will go toward developing the methodology for a world-first framework that will make it more difficult for attackers to identify specific individuals from raw data.

NSW Minister for Customer Service, Victor Dominello, said the efforts in this directed ideation was crucial for the future of Australia.

"People ask me all the time what are the biggest challenges you face as a minister in trying to get the government to the point where it needs to be in the next three, five or ten years,” Dominello said.

“I say there's two things – one is I have to get the culture right, and the second thing is I have to ensure there's trust in the data.

"So when we do things that are big, bold and visionary around data we have the trust in the community.

"This is so important to all of us."

The Freedom Fighters took home a $2,800 prize for their efforts.

While big data has many uses for evaluating trends and making decisions, one of the biggest risks when publishing or sharing raw data is that it can be misused to find specific people.

Researchers from the University of Melbourne found ways to identify patients based on public health records released in 2016.

NSW Chief Data Scientist, Dr Ian Oppermann, has been working hard to mitigate the risks posed by big data, but he said there was still a long way to go before this problem is completely solved.

“We had teams throw incredible intellectual effort, a lot of computing and quite a lot of really creative intelligence against a problem which is complex and subtle,” Oppermann said.

“We turned the dial another half-a-turn and that has helped inform a lot of what we will do going forwards, but clearly there's a lot more work to be done."

During the judging of presentations, the panel bemoaned strong critiques of their previous work, but welcomed the challenges and accepted them as the necessary – if not emotionally difficult – parts of science.

ACS has previously hosted hackathons aiming to solve this data sharing problem and ACS President Yohan Ramasundara welcomed the extra efforts on this endeavour.

"Without a secure framework, private information is always going to be compromised and the collective value of data will be underleveraged,” he said.

"For all governments, including the NSW government, this is a high stakes game of risk and reward.

"ACS through this directed ideation has brought options, subject matter experts and decision makers around the table to consider the next best steps."