Senate Economics Legislation Committee, , 7:24:28 to 7:53:54
The AI Safety Institute discusses its role in investigating the Services Australia data breach and testing frontier AI models.
Topics: Artificial intelligence, Privacy, Procurement
Speakers: Senator The Hon Jane Hume, Helen Wilson, Matt Crawshaw, Speaker 39, Senator The Hon Tim Ayres, Julia Pickworth, Senator Lisa Darmanin, Senator Steph Hodgins-May, Simon Draper, Speaker 40
>> Okay, welcome back everyone. Um, we are continuing on uh outcome 1.1 and
I'm going to Senator Hodgins-May main
Senator Hajins, may I will just confirm for everyone who was following on earlier, we have agreed to table the letter that you have circulated. Thank
>> I'm just waiting. I said I I'll um call on the same >> Thank you. Um after taking advice during the break u for us to resolve this issue of claiming confidentiality which doesn't bind the [clears throat] Senate um you need to claim public interest immunity which the committee will consider. So I'd like to ask is that what you're intending to do claim? >> I think we'll take it on notice Senator. >> Take it on notice.
uh not straightforward propositions outlined in the correspondence uh that
intersect. It's fairly straightforward
>> between the between the um the the powers of the Senate and the committee which are um wide and large uh and the interaction between those uh and the provisions of uh the various pieces of legislation which are described and then a set of um the letter helpfully sets out a set of public interest immunity grounds that you know we will consider did um within the broader remmit of our responsibilities to uh you know we'll have an eye on the confidentiality requirements for uh individuals and firms who uh provide data to the uh tax office and to the department. So we'll just take them on notice and if we can be helpful we will.
>> Okay. And as mentioned, this is something that looks like it will be need to be more imminently addressed through the R&D legislation if we're not going to get the answers from the committee today. Thank you. Thank you, Chair.
>> Thank you. Um, so we're going to Senator Hume, but we just um do you want to um check which outcome your question
>> about the R&D tax incentive again? So I assume actually I might have waited into 1.2 two accidentally, but
>> I think we can manage it. We're the right people. Yeah. And we've got the
>> Let's confirm with everyone. We are now moving then into program 1.2. Um
>> so we can we can we can cover our R&D tax incentive in outcome 1.1 or 1.2. It sort of depends a bit on the sectors. So if you if it's particularly about the critical technology world, then that's probably better in 1.2 because we'd add our colleagues from 1.2 to the table. But we will be here regardless. Hang
with me for a few more minutes and then we'll we'll finish R&D and then move on to AI institute. Once we've done that, we'll say AI safety and then we're then we're off. Okay, cooking with gas. So, budget paper two says that the R&D tax incentive reforms decreases payments by 1.6 6 billion over 5 years. And budget paper one on page 175 records the R&D tax incentive reforms as savings of 650 million over 5 years. Is that correct?
>> Yeah, Senator, I'm sorry I can't come in on that. I You'd have to check that with Treasury. I can give some general advice that uh the R&D tax incentive provides uh a refundable offset and then a non-refundable offset and they are treated differently in the context of budget paper preparation. So one is an expense, the other is not. So it can create
>> right
>> some figures that don't necessarily leap off the page as interconnected.
>> Okay. Well, riddle me this. A $650 million saving means that the government's going to spend $650 million dollars less supporting businesses R&D than it otherwise would. Is that right?
>> So, Senator, those figures in the budget papers would have been prepared by Treasury. What um Mr. Craw is happy to do is kind of explain the mechanisms of how we administer the R&D tax incentive if that's useful in understanding. I I'm
just not I just can't understand how you can have a $650 million saving and then say that the government is significantly improving its investments in R&D.
>> I could talk in general terms about that. it the it's about a better targeted scheme and so uh targeting the offset to particular cohorts of firms or sectors or sizes where you're likely to see more additionality of the R&D spend for the private sector. So it is more an exercise in trying to get more R&D spend per taxpayer dollar rather than necessarily the total tax
>> total amount of dollars. Okay. So it it
would go to elements of that but again Treasury would have to cover the details on those assumptions that underpin the modeling.
>> Okay. So I think probably more up your alley then is whether um when companies say that they are reviewing where they're going to locate their R&D tax their R&D activity that's something that you would monitor whether R&D activity is being attracted to Australia or whether it's moving offshore. I would
suggest us and treasury very much are interested in the in scheme attracting R&D investment that that is what the the reforms are designed to achieve.
>> Okay. So do you track that?
>> Uh yes we have facts and figures on participation through the programs publishes R&D figures.
>> Okay. So where do we find that? We do we can is there somewhere where we can sort of transparently see whether R&D activity is happening more in Australia
ABS publishes the uh an R&D catalog
>> and is that is there a correlation between ABS's R&D catalog and the R&D tax incentive and the use of the R&D tax incentive?
>> Uh not in those ABS figures but for the R&D specifically the ATO publishers tax statistics uh they are able to be searched for RDTI facts and figures.
>> Right. How do you marry up the two? Do you is that what you is that what
>> well they are they are data sources that would be assessed periodically to see which how they're trending and they would be matters that feed into modeling and program design considerations. Is
there a close correlation between changes to R&D tax incentive and the data that the ABS is that you've referred to as to where R&D is
>> The the I probably did take that on notice for details, Senator, but I I just observe generally that the ABS's definition of R&D is a bit broader than what is eligible to be claimed for the offset within the R&D tax incentive. So the two are not like for like >> um but the the macro investment in R&D is something that's important indicator for the economy and that's factored into R&DTI and broader R&D settings.
>> So before the budget did your department model the impact of removing supporting activities.
>> So Senator um uh in line with what we talked about last estimates um uh the the the budget uh the budget preparation means that we're limited in what we can talk about in terms of our engagement in the budget process. Um what we absolutely can do as we've done today is talk about our engagement in the consultation and the design and and how we administer the program but um questions that go to kind of the actual uh the budget process itself we're limited in what we can
>> Okay. So I'll just ask the question again then and perhaps You can say yes or no. Did department model the impact of removing the support activities?
>> So Senator um Treasury [clears throat] um led the work on this through the budget and as part of the budget process and so we're limited in a what we can answer given Treasury led our budget process
>> in modeled the impact of removing the
supporting activities. So, Senator, I'm I'm what I'm saying is that um we don't normally go to the specifics of what is u part of budget process and cabinet processes um and that this measure as we've said today was led um through rightly given tax um through Treasury.
>> Okay. So, so who is it that then provides advice to government on the effects of removing supporting activities before the budget?
So, Senator, what uh what I'm saying is that as part of the budget process um uh it's obviously a cabinet process um in line with sort of usual practice, we don't reveal the details of budget and cabinet processes. Um this measure is one that has been led uh by the
Except for the fact that this is kind of an important element and it's your department's area of expertise. So has there been is there modeling of current registrants who are going to lose more from the removal of supporting activities than they than they gain from the um from the core uplift the percentage point core uplift.
So Senator um uh my colleague can certainly talk to the modeling that Treasury has released as part of this process but um further questions around modeling should be directed to the
>> Okay. So companies have told us that the removal of these supporting activities, the removal of supporting activities is going to cut their refundable R&D tax offset by up to 60%. And I'm wondering if the department or anyone else has tested that claim against its own
>> So s we we uh we do analyze our RDTI registration data and we support the treasury modeling with provision of data. Uh we're not working on the modeling to be clear. Uh there is also data that is provided through the ATO as well. I just want to stress this department here. We receive registrations for the R&D tax incentive upfront, but that doesn't necessarily mean that that is the exact amount that's claimed later on through the company's income tax statement with the ATO. So figures on actual claims that are verified is an ATO uh responsibility in the co-administration of the scheme.
>> Yeah. Okay. I suppose this is a you know a really important element of our tax system. It drives productivity when done right and I worry that it's being changed without any mechanism to assess effectiveness. What is the me mechanism that your department puts in place to assess the effectiveness of these
>> Well, Senator, we would be evaluating how the program is implemented and the reforms are implemented. We would be tracking metrics uh on those factors. Uh once the reforms are uh if if the parliament passes them, they will be implemented and monitored.
>> Okay, that's exactly the answer I was looking for. Can you please on notice provide us a list of the metrics that you will be looking at to assess the effectiveness of these changes and over what time frames those metrics will be considered? Okay, chair. I'd like to move on to the
so that's it for 1.1 and 1.2. So, yeah,
that's 1.1. We're 1.1. So, that's it for
Is in 1.8.
>> All right. Thank you to all the colleagues from 1.1. Uh, and we will now invite um the Department of Industry Science Resources for program 1.2. uh along with the uh where I'm office of industry innovation and science Australia Australia's chief and Australia's chief scientist and I think we also have yes the AI safety institute please and we'll just suspend for a short moment while everybody takes their faces. Okay. Um, the committee welcomes representatives from the Department of Industry, Science and Resources for Program 1.2, Investing in Science, Technology, and Commercialization, the Office of Industry, Innovation, and Science Australia, Australia's Chief Scientist, and the AI Safety Institute. Uh, so I'm not sure if we have any opening statements. If so, they are tabled uh in the interest of time and we will go straight to questions. Senator Hume.
>> Thank you, Chair. Uh I have some questions specifically for the AI safety institute. uh when the government first um did a media release on the AI safety institute in November, it said it the AI safety institute was described as the government's hub of AI safety expertise and that its role was to evaluate emerging AI capabilities and support timely action on risks. I just want to make sure that that is still the remit of the AI safety institute.
>> Thank you. Senator Helen Wilson, Deputy Secretary, Science and Technology Group. I can confirm that is still the goals of the safety institute. They really have three goals. That is to analyze and test new AI models and applications, provide technical support to agencies and regulators in responding to emerging AI capabilities, risks and harms. And they're also responsible for helping shape safe I development, deployment and international governmentance in Australia's interests. Terrific. All
right. I I have some questions specifically regarding the open AI agent that ca um u unauthorized access to the Medicare statistics reporting service portal first and foremost. Um obviously the first Commonwealth agency to learn of that was Services Australia itself on the 11th of September. I I'm I'm assuming that the AI Safety Institute has been either directly involved or at least playing along at home on this of this on this issue.
>> Yes, absolutely. The AI safety institute did assist services Australia and ASD.
>> Excellent. All right.
>> So, principally the um cyber security element of this which is a is a very large part of this is ASD of course.
>> Yes. Um, so Service Australia not notified ASD on the 15th of September. What was the date that the AI Safety Institute was first informed of the
>> I can confirm that we were notified on the 19th of September. I'll just check with Dr. Kate Conroy who is here. Oh, >> would you like to come to the table Kate? Thank you. [gasps] >> 19th of September. That is correct.
>> 19th. And was it ASD or Services Australia or somebody else that informed the AI safety institute?
>> I will take that on notice, Senator, but my understanding is that our minister, the assistant minister for science, technology and digital economy uh was made aware as was our secretary
>> at the same time on the 19. uh between the 18th of June and that date, the institute obviously didn't detect or a flag or any um unusual activity on that Commonwealth system or any other Commonwealth system. Is that right? >> Uh the urgent review that was announced by the prime minister on the 24th of September is led by PMNC, not the AI safety institute. Um, has the institute been given access to the open AI model or the agent involved to test it?
>> Senator, we the the government has and continues to have discussions with AI developers, including in relation to access to frontier AI models. I'll pass to Dr. Connor in a minute. The way that we provided the technical support to ASD and Services Australia was making sure that we were asking OpenAI the right questions on exactly what had happened and I may now pass to Dr. Kate Conroy.
Thanks for the question. OpenAI has confirmed that the model responsible for the Medicare incident was an internal model that was being developed inside OpenAI and was not released publicly and was not made available as far as we're aware to any third parties for their
does the institute have any agreement at all with open AI to access its models in the future to be able to understand what happened here and make sure it doesn't happen again?
>> Yes. So the AI safety institute works with our technical partners including ASD and CSRO, right? >> And we do have arrangements for access to frontier AI closed models and openw weight models um through our technical partners as well as our own access.
Okay, I'm conscious of the fact that you guys didn't find out about this until over a week after services Australia. That seems a little lengthy. Is that
>> Well, we responded to ministerial direction with regards to when we uh responded to the incident uh when the incident was made available to services Australia through OpenAI. So open I contacted services Australia in the first instance
>> on the 11th of September.
>> That's that's right. Um
>> you guys found out on the 19th. So the
way that the instant was first understood was primarily through a traditional cyber security incident framework and my understanding is that services Australia and ASD conducted an investigation of that as a cyber incident at the time and so through the course of that uh exploration and understanding of the incident they then reached out to others including the Australian AI Safety Institute in order to then better understand the nature of the incident. and to draw on our technical expertise.
>> Okay. I would have thought that you guys would have led the uh inquiry into this rather than PMNC.
Senator, the rapid review that you're referring to, which is the responsibility of the department of prime minister and cabinet, >> is looking into the arrangements for AIdriven cyber incidents. Uh not only the services Australia one, but it is thinking about what might happen next and how we coordinate uh among relevant Commonwealth agencies. We are working closely with PMNC with the national cyber security coordinator with Australian signals directorate as prime minister and cabinet conducts that
review. Okay. So, the government's self-proclaimed hub of AI safety expertise learned of the hack of an Australian government system a week after services Australia, after ASD, days after ASD, after the finance minister and it's not leading the
>> So, the the the cyber security questions are principally Australian signals directorate >> questions. Um there is of course an overlap here uh with the AI safety institutees work and the scope of work that Dr. Conroy and Miss Wilson have um taken you to um the not not to confuse the sequence of events here with the rapid review that the prime minister has commissioned prime minister and cabinet to do which will go to you know that set of questions is there uh a requirement to um you know to look at the processes and the alerts here. It was very clear from when the open the the way that the open AI instit incident was initially notified >> to uh the government was unsatisfactory that doesn't abregate of course the way that we uh manage uh these issues across government and between the agencies. So, it's been a good opportunity uh to engage those questions and we'll have more to say about that in due course.
All right. Um the date of that first media release I think I said was no um 205 25th of November 2025. Um what was the institute's formal date of commencement of operations? Was it was it that date or was it just after
>> The 1st of June 2026.
>> June 26. Okay. Thank you. So on the 6th of October um Anthropic told a parliamentary committee that it was still finalizing an agreement for the institute to run its own independent evaluations of its models front of its frontier models. Um so I just want to be very clear here. Does that mean that at the 6th of October there was no executed agreement in existence that allowed the institute to independently evaluate anthropics models? Is that right,
Senator? We the AI safety institute does have anou with anthropic and we also have one with Microsoft both of which support technical exchanges and collaboration. We are just working through some of the more technical aspects of how they would then go to share um access give us access to frontier models.
>> Okay. When do you expect any agreements to be finalized?
>> Uh so I've um been in contact with Anthropic over the last week and uh they're working to coordinate um the advancement of our agreements. Uh but in the meantime we do have our technical partners that we work with to access anthropic models.
>> Do do you have any executed agreements [clears throat] with any other providers any other frontier models? >> Google Deep Mind.
>> Yes. As as uh Deputy Secretary uh Wilson said we have with Microsoft and Anthropic in place at this time.
So I'm sorry. Is anouou in from your perspective anou operates the same as an executed agreement?
>> I will take that on notice. Theou is absolutely to support technical exchanges. >> I can bring up a copy of theou. And at this point, I'd also note we have excellent relationships andus with our Canadian uh counterparts with the UK's AI security institute. These arrangements andus also support information sharing, joint research, how are we all undertaking testing and evaluation, how are we sharing expertise. So the institute has a range of ways that it can think about access that it can think about evaluation that it can think about safety and security. So it's not just through the arrangements that they have with frontier AI companies it's also with equivalent safety security institutes around the world.
>> Okay. So in July uh the assistant minister Mr. Charton said that the institute was, and I'm quoting him here, testing the most powerful models on the planet. Um, as of this month, the only lab with anou apparently told a parliamentary committee that the agreement to let you test its model is still being finalized and that's been confirmed today. Obviously, both statements can't be entirely accurate. Was Mr. Charlton being a little expansive in his description of what it is that you are doing right now?
Simon Wright, head of technology uh and digital policy division. Senator, um it it is the case that the AI safety institute tests models uh and it does that routinely. >> However, I think what you're alluding to is early access to frontier models uh that the companies would make available pre-eployment. Yeah. And that's a matter we're working through in relation to
>> you haven't actually specific
arrangements
>> tested any frontier models before they're released
>> yet. But as Dr. Conroy said, we have arrangements through the Australian Signals Directorates and uh uh other uh government bodies to access models through their mechanisms which can include uh pre-eployment models.
>> Okay. So just to be very clear, have you actually tested any Frontier models before they are released?
So I might speak to what I believe assistant minister Charlton was referring to at the time that he spoke at the AI safety forum in July. He referred to uh testing we did with our technical partners of Frontier models in June of this year within our first month of operation. He was correct in stating that we were testing some of the best and most powerful models at that time in that statement.
>> Right. So not independently but you have
partners we work with. It was our first month of operation and we were able to work with the Australian government in order to progress those tests.
>> Okay. And that was a test before public
>> Uh that was a test of Frontier AR models with our technical partners. Um and I might take that on notice in terms of clarifying the specifics of the models because our technical partners are part of that agreement.
>> Okay. That sounded like a very careful way of answering that question. But I will trust you that when you take that question on notice and give me an answer, it will be something more wholesome there, but I won't put you on the spot here.
>> It's also the case I mean that that that exchange is perfectly right. I mean there is more work to do here to deepen >> the the engagement. One of the reasons of course that um the AI standards legislation is so important uh is that in our view uh uh if um Australia becomes the destination for training investment uh it does um provide for an enormous opportunity for us to deepen that commitment uh that cooperation I should say.
>> Okay. Um can I just follow up very briefly with just an understanding of how many full-time equivalent staff the institute has today?
>> Uh certainly uh the AI institutes sorry AI safety institutees headcount is 29. So that's uh 27.4 FTE
>> right and how many are technical roles like research science or engineers and how many are administrative?
>> Yep. Uh we have seven staff working on evaluating models >> 6.4 for FTE and there are six staff in the regulatory engagement team uh 5.8.
>> Okay. And I should just double check, Dr. Conroy, you're still the responsible AI lead at the RAAF.
>> I am still a reserveist for the Royal Australian Air Force.
>> Okay. Thank you for your service. Um and I should ask what proportion of a full-time load is for you in in the institute as opposed >> as part of
>> I am full-time general manager.
>> Okay. Terrific. Thank you very much >> I think we can move on.