The Distinguished Colloquium Speaker Series brings national leaders to the IU School of Public Health-Bloomington throughout the academic year. These events, which are free and open to the public, highlight key topics and contemporary issues in public health.
Check back for more info as the 2024–25 series expands!
Jasmin Divers, Ph.D.
Director, Division of Health Services
NYU Long Island School of Medicine
"Methodologic Approaches For Diabetes Surveillance Using Electronic Health Record Data"
October 23, 2024
11:30 a.m.
State Room East, Indiana Memorial Union
Description of the video:
Good morning, everybody. You have you all here today for the followings of our distinguished colloquium series. I'm just seeing it, so welcome to you, I want to introduce momentarily a School of Public Health. Before I do so, just ask you to please keep your fumes on silent. The event will run an hour and a quarter. We will have time for Q and A. So without further ado, p announce me. Well, welcome, everybody. I'm grilled to be here today. I have a very good friend and a good colleague and a good speaker here. I won't do the formal introduction as usual. We'll pass to doctor Lee, do that. But I'll say a few remarks about doctor Div and doctor Lee. Jasmine Diva, many people here of Jasmine and they're waiting to meet her. But this is Mr. Doctor Jasmine Deval. And I got one of those wonderful phone calls that I get every couple of years about 20 years ago. And my friend, Derek Horton, who some of you met visited here as a distinguished leer. Paul reed sit back in Alabama, and he said, is someone you need to meet. This person, Jasmine Devil, is graduating under Steve Finch, who is a well known statistical mathematic geneticist, Stoning Brook. This guy is Shark. You should hire. And if somebody like that calls says you should hire, I almost always say. Get him on a plane, you get him right down here right now or her. You've got great people that way. And so We met Jasmine and we hired him and what a great decision. And I think, it's interesting I was talking to another friend of mine, Arthur Skipper Ba, some of you know, the Vice President of research at Michigan now. And for a while he was at NSF as a rotator. Successful being a very high level. What made you so successful, Skipper? And he said, What made me so successful is whenever something came up, somebody asked me, what should we do? H I didn't think about myself. I just said, What would help you organization, and I would say to that. What's helpful? And that's what I would offer. And I think that's what Yasmin offers. What Jasmine says is, how can I make an impact? And he does that through the intellectual rigor of his work. I've always been impressed by the conceptual ring. He won't rest until things are clear. Wants it to clear, wants it make sense. Precise. But he's also willing to be applied. He's not someone who says it's just me and my theories. G into the application. Let's make a difference. Let's benefit. And now he's also an academic leader. We're just talking about being academic leaders and got intellectual part, social part, business part, so now you have to be there for so many people. And it's a high level responsibility and obligation to be there for those people. And I know he has that commitment. So we've had great fun together over the years. Intellectually, socially. One great time we had was So years back, I had three different post Tocs, one spoke French and one spoke Spanish and Italian and one spoke English. The English one we took to England for a week to write a grant proposal with Jonathan Flynn and the Spanish one and speak Italian to Italy. And Jasmine and I went to France and we spent a week writing a grant proposal and it got funded. And it was to look at statistical genetics of that. And so he's always about the ideas, the rigor to make it clear, and I think that's what got all so always. So welcome. And now let me introduce doctor Bing to you. We do the formula right. Thank you so much for joining us today. It's my great pleasure to introduce doctor Jasmine Deer. And I also want to apologize. I just realized I've been calling him doctor Divers for. Sorry about that. Yeah. Doctor Des is visiting us from New York. He's a ion professor in NYU Grossman, Long Island School of Medicine, and he's also serving as the division chief of ger. The Division of Health Services Research within the Department of Foundations of Medicine. He also plays a leadership role in many other capacities. For example, he's the Associate Director of the Institute for Excellence and Health Acuity, and also the Director for Center for Health Services Research and Population Health. Before he joined MU in 2019, he was already a ion professor at Wick Forest University School Medicine, where he actually joined as assistant professor in 2007. Okay. And on the research side, doctor Deer had his PhD degree in applied math applied math and statistics in 2004. And then he completed his post doctoral training at Unis Obama Birmingham on Systico genetics, and that's with Ellison. So he has extensive experience in many areas of research. So for example, he has published about 2:50 papers so far and also served as the principal investigators Michael Founding ground on kidney disease and kidney disease, kidney transplantation, type one diabetes, type two diabetes, cognitive function, and aging. I can go on about his achievement that's going to take a while, but I'm going to stop here so we can enjoy stop. Please join me welcoming doctor Divers, stop. Thank you. T hank you for the great introductions. I remember that drive from Long Ind, New York to Alabama. 22 hours. And the culture shock getting in Birmingham, I remember going to the DMV at 4:50. In New York, if you get to the DMV at 4:59, you get they close the door. They serve you. I go to the DMV in Birmingham, there was about 3:30 and the lady comes. You get to the white place. But I think we have enough people here to serve to keep us busy until five. Come back tomorrow. So I remember this times fondly. So thank you. Thank you all for spending some time with me and for giving an opportunity to talk about the specific project. So the project, as I'll get into it, is a surveillance study. I was a great question being last week. Are we doing this as a research project or a surveillance project or as both. And I think the way we've gone about it is to do both. So if I can get to the next slide, I'll talk a bit about the not so grammary aspect of the large multi center studies. And then I'll get into more of the methodologic issues and some of the questions that we're wrestling with. So the IFA is sh that's not going to take that long, but we get going when you start talking about the soft scales that take to get contex like that off the. So the RFA simple. It's a simple request from the CDC to put together a system, mostly based on electronic Afer data to build a diabetes suban system in kids and young adults, as we call it, 0-4045. So what's new about the RAFA that we've done surveillance studies before where you go to the records, you make sure you count every single case of the outcome of interest every single case, every diss case. And that's how you be registries. And that's good work. That's essentially doing the census of all cases in the geographic area. And that can take some time. And what the CDC wanted to do with this RFPs to see if we can go faster by using the already collected and available in Hs. So they put the RFA out. I will never know how many people applied, but I know that eight centers got funded. And you see the centers here. That's the City call. So I don't know why it just happened to just line up in the center of the country. But that's what happened. So eight centers p is small, but we can see it on the side from Gazana in Pennsylvania all the way south to Florida. There was actually one a couple of centers in the south. And then the rest is in the center from east to west. And and then we were successfully funded as the data according center. We're happy, we got funded. It was a tough competition. Now we have to do the work. The first thing that we decided to do is that we have to make sure we're all speaking the same language, that we are calling the same outcomes by the same names, that we're coding of variables the same or systematic. Well, it turns out that's a lot easier set than that. Because the way the study was funded. Some of the sites ended up being all epidemiologist and the epidemiology, then epolgy, top top bottom left and right. Some of the other sites were computer scientists. And they have no idea of very little idea of epidemiology and how to put the studies together. We had to get them somehow speaking the same language. I remember in one of the earlier calls, one of the computer scientists said, well, we have to set our ETL processes within seconds, I got like 20 e mails. What the heck is ETL. ETL is computer science pick to say that we're going to extract the data. We're going to transform it in a way that makes sense. And then we're going to load it into a system. So it took us a good three months just to get people to sort of think about the same problem because they came in with very different ideas of what I need to do. So by the time you lined up the experiences, the experience of the different groups, we lined up the different data sources. I have a slide on data sources and get people to stop thinking more about what's good for my side as opposed to what's good for the entire network. A year went by. And I went back a last week looking at the proposals. And every single proposal I read said that we will get our protocol together in six months. And it took us a year to just speak the same language. But what typically happens in these big studies is that you have to put together the structure of the study. You have to put together what's leadership? What's the steering committee? What's the publication BD? Who has what expertise to serve into what committee? And that that takes time, that takes skills. And that takes I don't want to call it politics, but some of what it is. And this is not the stuff that I learned in S 500 Tony Book or the stuff you learn in the epidemiology course. This is actually sitting down together and listening to different ideas, different objectives. And everybody wants to be represented, everybody wants to be able to show to the funders that they're doing their part, but getting to a consensus where we are all happy and feel that they're part of the same system. That that took a lot longer. Tan I think anybody thought at the beginning of of of the network. So to give you a brief overview of what we mean by having the different data sources. So we took that survey, see the results now on the screen from one of the sites that were funded. And I just asked them, do you know where your data come from? Yes. Do you know how many sources you have? And you can see the numbers here. Sometime and 18 different data sources. Some of them already had them combined into a common data language mo that they use. Some of them just have seven different health systems in their state that they have to some somehow combine together into 11. And we know that we wanted to go to truck validation for outcomes. One of the questions was, well, you have the systems, but how many of them can you really access? Can you actually send somebody to pull up the record and look at it and then make a determination. You can see the numbers drop out from 58 to about half Then we ask questions about other data sources. It came it became ef very quickly that A, we have to build a common data model for the study. B, we have to now get the lawyers that's my next slide. You start talking about how we set up the contracts for this data to come to s. So we ultimately had that in place. We had a common data model. It took some sites of the more computer science health informatics heavy sites. Maybe two, three months to get that together. It took some of the sites that were led mostly by the epidemiology that knows everything about diabetes epidemiology, but not very a lot of computer resources. It took them ten months to a year. Even now, we have five year into the project and they're still asking, was that we necessary? We would not have use the systems that we have. And get those data together. But anyway, so that was the first 15 to 18 months of the project, just getting the data use agreements in place, getting the common data get making sure that data can move from one side to another and that this data will be stored in a way that is safe and that we're not going to try to identify who the patient is and so. So that's I wanted to have this to give you a brief overview of what it is ready to do the sort of large scale multi center studies, especially outside of clinical trials. I think clinical trials are a bit easier in that sense because you have a clear protocol. Typically, you're not collecting data on millions or hundreds of thousands and millions of patients. It's a little bit of different scale for the sort of intral based multi center studies. So what did we learn? We learned that it's good to have common approaches. It's good to have common data models in place. That builds efficiency. That makes it easier later when we start to get into more the epidemiology statistics. Aspect of the study to be able to combine results to compare and understand sources of erogenD, having everything the same way helps. And unlike a clinical trial protocol for a clinical trial, that is this almost rigid thing that you cannot change very easily. For this sort of studies, it ultimately became understood that the protocol is just a guideline that it's a living document that will have to be amended, and we have amended ours multiple times a reading as the study is progressing because we're finding things that we didn't know at the beginning of the study. Okay. So that's the first part. We can do big data. We can do this sort of large collaborative work across multiple institutions, but it typically takes a lot longer than we think. Typically we need to allow sometime. But the project itself, I think about it sometimes I say it's a very simple project. We just need to estimate a proportion. We just need to know what is a numerator or the number of diabetes cases. What is our deuminator? What is the population of ves? That's what we need. Maybe since we're using HR data, we may have to do something to make what we estimate in the HR somewhat relevant to what we see in the population. That's where bias correction comes in. It's really three components is finding out who has diabetes, based on the HR system, finding out which population did they represent and making sure estimates represent that population. So on the face of it, it's a very simple, very simple part. So as part of the protocol. And what typically happens in the Mot center studies is that in our case, we nine independent projects. And then once you get funded, the funding agency comes in and say, congratulations on getting funded. Those are nice are not nice proposals that you. Don't forget about them, but put them aside. And now come up with the protocol for the entire network. So you have to essentially sit down and redesign what we're going to do now as a network as opposed to the single project that you proposed to. Ultimately, the network as opposed to the part specific that is we need to define our computable phenotype, to be able to tell looking at t data, who has diabetes, who has type one diabetes, who has type two diabetes. And in which case is accident versus p. And then once you do that in A two, then you can do what we really want you to do is to get those estimates out. Then three, which is a larger question is, how do we then go from what we've learned in this experiment to the rest of the country. How do we build a sustainable diabetes surveillance system using the HR? Those are the three network games. But again, remember, it's a very simple problem. Find out how many cases, find out what the dominator is and find out what the population is. One of the first question then, what does diabetes look like in the EHR system? So what typically happens when a patient goes in and see primary care provider is that you go, you get your labs, you go, you get your test done, you get everything done, you come in and have a visit. And maybe the physician goes into the note and say, well, patient has the ID. And that that is somewhere outside of what's called the structured data where your lab data goes where the drugs you want Go. So if you want to build a system that use essentially what is entered structural in the system of graphic and port, what is in the structured data. Then you have to be able to tell when somebody has diabetes because that narrative language is not necessarily in the structured data in the HR. So we have to be able to say then what is diabetes in the HR system. But I have diabetes. It's not just diabetes. So how do you distinguish someone with type one diabetes type the diabetes in HR system? So you have to sort of have algorithms to figure those out. So we call those algorithms computable phenotypes. Sorry. We call them compatable phenotypes, that is an algorithm that you can execute an HR system and come up with an algo. What I have here is on the top is what our idea of what diabetes will look like in an EHR system. We call it Day is the one of the study and we call it the comp phenotype. What we said is that it's true that we're not going to have a a diagnosis spelled out in the structured data elements. But when the patient comes for a visit, they're going to be charged, whatever the health system is, they have to charge a health system. And to charge a health system, they have to provide sign a code, what's called an. Diss classification code. So we said initially that we're going to call somebody a diabetic. If they were hospitalized, so they had an inpatient visit, at the end of that visit, the provideer bill insurance for diabetes. That's an inpatient visit. Or they had two other encounters, not hospitalized, but at the end of those encounters that they were billed for diabetes. So we said either one inpatient diabetes code or two outpatient diabetes codes. But before that, before this project came along, there were other diabetes combo phenotypes. And I have this is not exhaustive, but I have five main categories of com phenotype of diabetes. There was one paper published right before the IFA came out that says, you only need 11 code. Number five here that's highlighted, it's a paper that I worked with Barn Wells at forest. That says, Well, it looks like you need two goods, independently of where they come to be able to tell des other diagnoses in the health system. What we have in between are some of the same definitions, but they require a. For example, number two says that they need to have a diabetes. But they also need to have prescription for. That is probably looking for i T is probably and four very similar symptom to take most space on the side, that is, they have to have the codes. They have to have evidence based on the lab, that is, they have a one C measure, a blood sugar level that suggests diabetes, and that they have other measurements, either that they have high codes or they are taking diabetes medication and so on. And same for four. So you could see that maybe one, two to seven extent and five very straightforward. And three or two and four require more prescriptions, labs, something more. And it became very became evident very quickly that three and foward is not pic because not everybody has labs. And then if you don't have lab, then what happens? If you have a diabetes code, but you never have a lab, then you require you diabetes, diabetic code, or that We ended up pulling away too many potential cases because we didn't have other data elements beyond the ICD ten codes for decisions. So what we ended up doing then is comparing the three simple alternatives for the computer po type, the one we had proposed initially, we call it the ICP in the next set of slides. The one on top, we're going to call it D and the one at the bottom, we're going to call it well. But before we even do that, we had to plan for a validation that is we're going to go into the chart. We're going to pull a subset of charts, and we're going to actually look at what the ito type is saying and compare it to what the chart obstructor is seeing, looking at the entire patients chart. So what's on the left side is what I've b out on the other side is that first of all, we needed to get a sample of we call them presumptive ats case of people who look like they may have dates in the HR. This was to make validation doable. Otherwise, otherwise you end up with a very large sample and very very few potential cases. So essentially wide you use a fishing analogy and say that we're going to take that big pond. We're going to try to get as much as possible in one little area, and then we're going to fish in that middle p. So that's a wide net. So if they ever had a high one C level, if they ever had a high up level, if they ever had diabetes for diagnosis for diabetes, or if they are taking drugs that are for diabetes treatment. So we're going to put all of these into a wide net. Then we're going to create a track obstraction form to allow the truck reviewers to go and say, Okay, I think this is the case, I think this is the case of type one diabetes. I think it is an inc case. That's what we did. Then as is the case in this large studies, if you're going to pull cases, you're going to need study coordinators or nurses or somebody to look at the job. There's a cost to that. The first step of any stiation doing this is that, we so only do this proportional size. But if you see what the numbers are, that's what I have under the Catchment area. We have some sites guys in with fur hundred thousand people. And some of the sites say University of Florida 12 million people. And they all have the same budget. So if you say that we need to do validation proportional to size, then the site with 12 million people will a how many people you want me to hire to do this. So what we end up doing is to say, let's all do at least a minimum. So all sites, you have the same budget. You do a minimum of 400. And then let's tuple the rest. Beginning of this such that we have some bit of better representation of the largest. So we end up in the end are the number of chart reviewed. So we solve that problem by looking at the budget each side. And then we told the chart reviewers. Go. First, find out if they are dames. And then among those that were found to have dames, find out if they have type one or type. And we still going to do this based on ICD ten codes. That is when the physician is billing, they have to be specific. So we are billing this patient because we take it tack on diabetes or because we take it type two diabetes. So they have type one specific code to a type to st So then we say especially in this population. Type one is still more prominent than type two. In cases of ts, go with type one. If you have somebody where you cannot determine what they have, then if it's not type one, it's t two and so. And then not the first time that is noted in the EHR, not in the in the EHR, such that we can tell when they have when they are incident cases. And then track other values for maybe other projects down the road. What was the one C or the under diagnosed, or was the of the tender diagnosis. So track those values and put them into a database that we create form. That's essentially what I just said, that's the logic of it. First, you apply the wide net, there are potential cases, and then identify a small pon from the small, you fabs and among those that you take outies, determine the diabetes that they have, and also determine when they became inc. So that pmy data came in. And this is small, what we could see from what the t reviewers were saying is that in the nk of youth population, that type one was more poon. So three to one ratio of type one to type two diabetes. And then in the older 80 to 45 H, type two diabetes was more poon. The proportions were higher and that In terms of age, there is a flip, as you can see in the type ones. I if you can see my mouse here, in the type ones here 522 type two, in the young age group 52 to 16, and then it flips for type two the other age. And then when that data finally came, we were unable to look at the performance of the different compatable phenotypes. This is a small and ended up saying that it looks like the sis algorithm. At has the best performance. So 94% sensitivity overall for the young kids, and then 93 for sensitivity overall for the 18 to 44. This has significant application for how you apply this for the pinotype in practice because it's very simple. You could even do this with claims data. You don't need labs, you don't need medications, and it's a simple query at any point in the past, whatever period you find for the past, the year, two years, whatever diagnose or was there any diagnosis of diabetes in that? And then we look at well, especially for type of diabetes dos adding insulin make a difference. And it into matamot cost us a little bit on the number of people with insulin measure. So we ultimately went for the simplest computable fine. Then we look at different levels of performance performance in the younger kids. You'll see that we clear 90% performance, specifically sci. PPV for this for this age group, maybe a drop in NPV value in the older kids, but overall very strong measure for surveillance. This is in the B group. Again, you see similar patterns above 90 with a slight drop in the older He group. For component. Coonent, A is usually mostly type one driven. Typ diabetes ultimately takes the patient to the hospital. You cannot have tapon diabetes and not be treated for. So we are actually doing well capturing cases like that. In component B where we take it mostly t to diabetes. You see the results start to fluctuate a little bit more. There is more it rot and AD because this is a young age group that necessarily go to the hospital as often, may not have as many diagnosis codes the pipe of diabetes group, you see Some fluctuations. And then that's a performance across different demographics. Again, similar results. Overall, we clear 90% sensitivity, but there is some variation depending on the demographic characteristics. That's what I was saying initially that type on diabetes for performance for como type work really well. You don't miss may type on cases because ultimately the hospital. Type diabetes, people could be working and have tap diabetes and not know it so the performance is a little bit. So this was for diabetes type. And then for incidents, incidence is interesting because for incidents, there's that question when did the patient develop diabetes. And to do that, you need to have them in the in the in the health record in the HR London. But that's so true for type of diabetes because in a lot of cases, it's the emergency that takes you to the hospital and that's when the diagnosis is made that you see a primary care providers I think you need to a specialist. And the first time they say the specialist, that's when the diagnosis. So what we realize is that if we ask them to be in the system for a long period of time, you tend to lose a lot of type one cases, or a lot of component A component A that is zero to 18 kids that are primarily type one. Type two is. Type two is older people anyway. So they have longer time to interact with the health system. For them, we say we need to make sure that you are in the system for at least two years without any evidence of diabetes. And when we see the first evidence, then we can say that's likely to be an inc. So we have that two years look back period. With this in place, you will see that the performance also varies with the number of cases we have component A, tap again up to 90% performance overall for Conent B. One of our sites was really small in Colorado, and therefore, the performance is a little bit lower than the others. Because you need to exclude all of those people who had two years of data. We did have twos of data before you can start start thinking about CTs. So what we see here is that performance can vary per se in dependit of quarter for diabetes in general type one of type two. It varies across different age groups. It a group that is primarily type on definition tends to do better, and it varies with actions. What's really important here is that coveto phenotype, we ended up very simple to implement convi phenotypes, which makes them a lot more portable, that can go to other systems where we don't have Lab data collected as well. You could do it cata. That's really Then to do incidents well, I think we still need to go back because incidence is smaller in terms of the proportions in the population. So we need a larger valuation study, which was not in the budget for this for this cycle, and that maybe we need to do other methods that natural enga processing to actually be able to do the physician notes to make these comparisons. So that part of the work is clean in the sense that all of that work would be done centrally. That is, we could build a database. We're talking about s 400 to 700 cases. So sites can bring that data to one place and that analysis can be done centrally. That's not the case for denominators. Who do DHR data will present? Because to do this, you need to take the entire system. And it became very clear that no health system was going to tell you send you the entire patient population and have that all NYU for us to do this. So that became evident very quickly that this work needed to be decentralized, D the sites, and then and then get the response back. So So we ended up saying that I don't think we're going to have a one side fit one side style fits all here, and that this work will not be done centrally. So what we did then is to work indefinitely with the site and see, how can you best determine your popile? Some of them were like, Well, we are util based population. That is, we can tell who's using our system. Based on that based on that, we can tell you who we represent. We have sites like gzinger and Cs that are membership based system. That is, you need to be a member of that system to get care in that system. There could be a loyalty based for many we've had how many people we've had continuously for the last five years in s. And that's going to be all And there are sites in between that. Well, we can tell you how many people in the population is seeing. So what is a market penetration? And market penetration is trick in a sense because it varies like a place like New York City with six, seven major hospitals. The penetration of in the market is going to be very small. A place that's more doesn't have to be rule, but a place where there's one big health system that everybody goes through. Penetration is going to be larger. So that's that's ultimately what we need of doing that is depending on the type of site, the knowledge of the local PIs of their site, that they come in and say, I think we do well in this area, but not so in this other area. That's the solution we ended up adopting. One other concern that they had is that we were dealing with the set n population here, especially the 18 to 45 population. That may not be the hospital every year. If you want to define a deinator and you want people to have interact with this system in the current here, we're going to miss a lot of case. So for those sides, we say, well, we need to go maybe if you want to do prevalence in 2018, maybe do 2018, and then but look back two prior years. And any value interactive in those two prior years, then the part of of the em. And that ultimately gives what we refer to as period prevalence. And we had another one of those discussions where the what the heck period prevalence, period prevalence, different point prevalence. That also then become another learning experiment back in to get this. And we couldn't say if you see 5% of the population then include that. So we couldn't say this is the universal color for the study. So it depends on sites were and all the population. Then how do you know you have the right answer? Because everybody defined them the way that works better for them. What we agreed to do is that, well, maybe we need to try and that you're going to get this estimate, you're going to get estimate from other surveys. You're going to get estimate from previous studies and see where we are we respect the studies. If we get the same results, that doesn't mean we're right, but we are getting something that is consistent if we get dent results, we don't understand what. I'm going to show you two examples. Actually, one example and then what we ended up with. So this is what one of us sites Florida sites did a couple of years ago. What they did is to say, Okay. So we know the dash line is a published incidence rate from the CDC search study, which they went for 20 years. And then what they say is, what if we vary color? That is what you see in the w in the x axis, that is what proportion of the Potion population is seen in the health system. What if we let this vary and then we make the cut of where we approximate what pops search. So that's what they did and they said that you need to see at least 55% of the geographic population in your health system before you can include the geographic population in Ultimately, we have denominators defined for all the sides. Then we have the comparison of this is what we see by demographic characteristics in the nation compared to what we see by demographic characteristics in the study. As you can see, it's more or less comparable except maybe at the bottom, where you start looking at renity, that the study ended up having a higher proportion of Asian and Pacific islanders, higher proportion of nonanwer portion of non Hispanic whites. What high proportion of So those are areas where we think that we may be overall underestimating providence or incidents if we wanted to provide a US level of estimate. Okay. So That is denominator. We've worked centrally we define other cases of and we've worked in a distributed fashion to figure out what the pocion is. And that now all we have to do is get to get an estimate. But one thing to keep in mind is that HR samples, HR populations of R samples or convenience. People have to complete the system. It's site, we are design a national survey where we know what the sampling rates are where we know we have this, we have a place to go and say, Okay, we're going to pay 20% of that population, 10% of that population. So we don't have sampling rates. It's who we get. And that has to menu. Even in the people that we do get, we know what we get is not good But there are people who are not in the system at all, that maybe are getting care elsewhere or are not getting care that we know we are not capturing at all. So that's a non probability sample. And reason that people pick one health system versus another, it's not always big, that may vary from one population to the other. It may vary for what type of insurance you have. May vary by whether you have insurance for that. And because we don't have the sampling ways, so we cannot do the traditional inference that we would do on on survey this So what we need up doing is is bias correction methods to adjust for those estimates that we get to the EHR to make them look more like what we will get in a survey study. We're going to borrow terminology from the missing data, literature to create parallels between the selection issue that I was describing, how people get into the EHR system to the missing data problem that we know more about in thee. We're going to say that if somebody is not in the EHR, so we cannot share what they are. Based on that we're going to focus on two particular potential selection process. Processes. One is selection at. That is almost equivalent to what we think of missing apodo. So given the data we have in the HR, selection into the system is independent of the diabetes. And then selection not atend that is when s when there is somehow relationship between the diabetes studies and the present or not in the. Then with that in place, we can use work that's done in surveys, that's done in sing data literature to address the dt. One particular approach is to say, I know how many people come into the system. I can say what my ization deator is, how many people are using are getting from this health system. I know how many people live in the same geographic area. And I can use these two numbers and I have the brief down. I just showed you a couple of slides ago about age rate and sex to say, what is the probability of being selected into the PHR? And then use that probability as if it was a sing rate, and then back correct my estimate to approximate the rate for the pbilation. That's one class of approach. The other class of approaches model based. I could build a model to predict the likelihood of having diabetes if you're in the HR and then use that to correct my est. That's the second class of approaches. So me is exactly that. So we have a breakdown of the population by age, race, sex, insurance studies, and so on in the LHR. We could create similar break breakdowns in the senses. And then use other data that we have on the census, like for a specific ZIP code for specific County, we have the STS survey, the American County survey that provides estimates on education level and income level. We can use what we know from those surveys from the specific areas to build a model to address the selection bias. So one class of mat models that do that work well is a multi level regressions certification. We take the data that we have by age sex and ethnicity. So we have this position. Let's say we have a G G cells like this. Then we can use a model to get the equivalence in each cell. That is, what is the equivalence of diabetes among ten to 15-year-old black females living in. We can create that model and get an estimate at that level. Then with that model then and then with the MLLP, I can then re weight those estimates to match what we see in the pop. This is done under the selection attend model. How does it work? We fit this in the patient framework where we have the number of patients for each of those groups. So it's a multi level model. At the top level, we have a mark. And then the second level, we have all of the co values that matured at that at the level of ventures. And so here, what we've done is out of the data of fix, all of the s on the X. And then we fit this model in the vision for mark. So we get the estimate for each for each ptition for each group, and then we are graded in the end. And that is what it is. We then get the proportion for if we want the overall weight of diabetes, we can aggregate all of the cells together. If we want proportions for a specific subset, then we can aggregate at the level of that subset. That is if I wanted to get an estimate of diabetes at the country level. So I aggregate all of the diabetes estimates for that country and that. So that's one approach. The other approach that we've used is to do to try to predict how many cases that we would see in a specific geographic area. The model looks very similar. But now instead of pting an estimate at a county level or at a specific subgroup, we get the model to predict for each person for an individual living for a girl ten to 15 living in this country in this. What is the likelihood that they would have and an ID? So again, under under son. So then the final estimate then becomes the combination of these buildings. That is, on one hand, I have the data, I can count how many cases I have with the specific characteristics of being in the group k comes on one person, so I can aggregate all of those. So this is the number of cases in the specific age group in the specific area. That's the first part of the estimate. That's observed, that's included. And then for what I don't observe, I can estimate based on the model, I can credit how many places that we will get in the in the same. And then the overall estimate then becomes the overall right become the weighted average of the two weights. And that's the. So what we've done here is that and that's probably the actual data actually from the study because the studies are published yet. But what we see here is the performance of these two models using the same data. So what's in blue, is a naive est. That is if we were to only use data from the HR, those would be that will be the estimate again. If we were to use the vis cored methods, and we get essentially the same thing we did two other graphs at the bottom. One thing to notice here is that this is very blue no, that is the cod rage the vis dected age or not that far apart, 0.29 0.29 and 0.26. So those are not that far apart. But The question that always comes up from this we see in projects like this is ultimately how many people have diabetes. That's what they want to know. Even more so than what provide mean that we need to address for in terms of care for the population. Assuming those are the two estimates. Assuming that we have to do this for the entire country, we're talking about a difference of 240,000 cases depending on which bicion estimate you. So we have big feelings about this on one hand, it seems to suggest that bicorion doesn't do much to correctly estimate. But that may be true. Especially at this level for the overall estimate, when you start to get into the more cost classified levels of age, sex, or essity, the effect might be bigger. But if that was only the estimate, then we think that when you get to estimating the number of diase cases in the population, that this could be make a difference. Then if you look at performance in other for the demographic factors, we see a little bit more production, socially more sex. But overall at this level, it seems to work it seems to work. But this is Mo data. This is Mo Data. Results seems to be consistent, but it will help if we could actually get a sense of what the performance is in in related data. What we've done, we use an idea. That that David and I worked on a long time ago, the idea of blast simulation because we didn't ti data idea. And we said, what if you were able to take data from one side and keep all other aspects of that data set tied and just change what the prevalence is. So we know what the two values are in the data. And and then compare this to modes. So we took data from guy the smaller site. So we spiked in the missing data, the missing data process, and then everything else remains the same, we assume the underlying diabetes programs of 2%. And then assume that we only have 17% of the population in the age. That is the system only 17% of of the patients that live in the geographic area. And then and then there's a differential rate of diabetes in the census population because maybe these people are healthier, that's why you end up interacting in the system as marge. So we made the diabetes in the population lower than what it is in the health system. And we do that number of times and generate this plot. So Gray is what we actually observed in the data. Red is what we got the plasma simulations and blue is right in the census. So in terms of demographic characteristics, more or less approximating the population in the census. Especially of the sample in the regular data. We're still matching the stances because of the characteristics of the of the area where we got this data. This is coming from Geisinger rural area, very wide population in Pennsylvania, and you'll see that in the original data and wide we observed, rates are typically higher, more likely to be wide than what is in the actual census data. But this seems to suggest that we are at least creating the same sort of processes we see the real in the real data. And this is what the as distribution looks like in the plasma data set. We put a 2.0 in the census and 2.8 in the population. So in terms of performance, this seems to suggest that there's a little bit slightly more biased identifying with the ma. But otherwise, but that is the predictive model where we try to predict the likelihood of. It is versus this model averaging MLP model. This is essentially where we are now in terms of the comparison. It looks like the more complex model that tries to predict the likelihood of having IDs tend to underperform. You see higher bias with the set of models than with an MLP. The MLOP is very simple to apply. We don't have to go to the prediction step. That is essentially where we are now. Our next step is to try to understand why we see this. I think one idea that could be, if I could go back a couple of slides, is that maybe here. Is that this method is much more complex. We have to to to fill out sly for the H distribution. We have s H and directions in the model. We think that maybe one of the reasons that ultimately the performance is low is because we are overfilling here. So one of our next steps is to go back, simplify this model, look at the model and then we do the comparison. But ultimately where we headed with this plasmid simulation is what happens on the Sno section is not at window because I have mixed feelings about that to to be honest, is that I don't think for example type of diabetes, we are missing that many cases outside of the health system because people get sick and have to see doctor anyway. So I don't think that they will end up in our health systems, but if we have our claims data set, we have other data sources we ultimately capture. But in other scenarios for tat to diabetes, for example, if it's all related to access to care. So there may be more diabetes out there in the population that we are not seeing in the health systems. But that's the next step of where we are. So I think that was my last slide. We we are in the midst of the study now trying to write the papers and study ff. We put together a sampling of simple to implement surveillance system. We a comp type that is just looking at diabetes codes. I think we still have some work to do in terms of how we define denominators for all sides and still have to work around the issue of sharing too much data because no side ever going to send you 20 million records for the patients for you to do research. There are other issues related to that. Then try to figure out what happens under section not do, which is of the work that we are doing we are doing now. With that, I thank you for your attention and your questions. Thank you so much wait about 10 minutes for questions. Two questions. The first one was I was trying to understand whether you looked at different possible sources of bias. What is it? I saw that and you looked at two different conditions, but there are also possible lo of sources of bias that contribute what you observed so I wonder if you can consider that. So one of them. That may be easy to implement is how do we incorporate the performance of the d itself into the estimate. That is, we have some estimates for the ss, which means that they about that s. But we could re our counts for that can be mathematics. I think that's your history. The ers are like not if we cite in the network as act. So it tee for us to see what is coming. But for those that have access to claims, on our next step is to say, what is the estimate seeing in compared to the estimate that we have see and the t those are two potential etes. No question one or question two. I train to understand the simulation. So the set up is to take the data, observe data. So essentially You keep all of the structure of that data a time. You say, there's probably some complex relationships going on here that I'm not able to capture that I cannot capture 20 variables, but we have this data. And then take that structure the way it is, and then use some of the d variables you have in that data and bs something when you know the answer, that is, if I know that agent sex are related to diabetes this way. Maybe it's a logistic model that has some rates for agent sex, and I know what the rage of diabetes is now in my data set. And how can I recreate this in my. So that's the overall idea. Hopefully, I get that right. That's We did a paper on this 2006, 2007 in the Kelly. Introduced the idea of class. The idea of real data manipulate some elements. I've tried to do those simulations for complex systems, and at some point you like I'm too s here for this sense. But this keeps a lot of the reality of the data, and you're just changing the tiny yow. Go. So I understand here that the estimate you were really interested in. You a least performance of the estimator there to bias. But do you have any idea how much variance you also in terms of PA, you have bias variance. So how much variance are you potentially sim. Yeah, that's a good question. I could say that that's more of a statistical view of it as opposed to what the funders want what the funders want to drive the research in the sense that how many people really have this how many conditions you're going to need to put in this area. That's why I want to make sure. If you say there's providence is factor in this area, that is truly res of what the area what truth is in this area. The notion of bias that's driving this study a lot more than what is all of these comparisons doing. The trade, we haven't much time. But based on what we've seen so far if you look at the great assuming that there seems to be small. Here the major here that dominator. Models, selection that the selection process is at window, is not at window. I mean, section at wind is easier place to start. Section not a wind is the next s. I think from the perspective of publication, what we are ping to do is to say here's the estimate in the siondm. But we've also looked at three or four different second wind mechanisms, and hopefully we get something that is compound. So that's like that. That is, that's what I tend to see that in this setting. This is the estimated window and then it's ation from that window, and we test for That's what I see. Because selection of that implies that you take an assumption of what the selection of the selection process is. And is only an assumption. Sp model the same assumption. Yes. The only deviation from that we're going to ass buses in. Yes. Why you Why did you know Because of the notes, we wanted somebody to be able to read the notes. If you were to do something that includes natural language processing, read the notes. You still have to be is energy. We wanted somebody to look at the notes and say on all of what seeing here, does this it and look for information notes that the physician also. At the end of some of the. But that some Yeah. That's what we need to basic because I think it's because the model is complex. That's my. So we're going to go back and to and that's my the follow up a follow up question. You use human views. Here you are meaning for the c control. It's for some of the pieces. It's input tab is input, for which purpose. So we did not just want to say We have this algorithm here that you're going to employ on the health system and you're going to go with whatever the is. W to be able to say to what extent is what the algorithm is saying, is matching what is. That was the main question. The other question for incidents, which you go recently has mated the truck. Even in the notes, the physicians are not going to w. We believe that is the incident case of the. They're going to say this patient here based on what I'm saying as. That's what we did for. But we want you to be able to say what is the date the first day when the physician put that in that's what incidence is my laptop because you have to make they don't have and that the year they have IDs is the year that you have to mat is times. It's defined. Roger, I know you have another question, but it's quarter tell. I want to be respectful everyone's time. So please join me once again, enjoy. In the bx. Bing want you come up and we'll get a picture together. So we have this inscription, or you stand in the middle. We have this inscription here and a very nice plaque for you. I'm not going to read the inscription because it's too long, print is too small. But it talks about your value as a scholar and how you help us make a impact in the real world. We're using the rigor of mathematical sciences. And I want to just say personally how grateful I am you. I was just fantastic. Okay. Can we get all picture here together? Perfect. A k. There, nice. Thank you, everybody for comment
Stefan Pasiakos, Ph.D.
Director, Office of Dietary Supplements
National Institutes of Health
"Limitations in Dietary Supplement Science: A Call for Greater Rigor, Reproducibility, and Transparency"
August 28, 2024
11:30 a.m.
State Room East, Indiana Memorial Union
Description of the video:
Thank you for coming and it's bed to see you. It is truly a privilege and honor and a special treat to introduce today's guest. And you know, these introductions, I don't want to read someone's CV. You can read it. I'm sure online. You can go to Link and find out where he got his degrees and all that. But I think giving a sense of the person as a scholar and we have a wonderful scholar, and as a leader, and we have a superb and as a human being, M. Good introduction. And this point is because this is an exceptional person. I've had the privilege of knowing doctor Passacos for at least a year or so now since he joined the NIH as the director of the office of dietary supplements in 2022. This is closer to two years. The field of science is one that many personalities make important contributions in different vantage ones. There are those who enjoy the beauty of exploring ideas, perhaps most typified by the mathematician, G Harding, who in his day was the preeminent mathematician who probably seen his character played by J Jeremy Irons in the movie, the man affinity. And he prided himself and wrote in a mathematician's apology that he had never done anything useful. And he thought that was important because he said useful mathematics is of I only look on things that are beautiful that are just about the ideas. It's actually not true that it was not useful. He came up with among other things, party win book equilibrium, which is one of the most important things used in the field of genetics. But it was so simple mathematically for him that he published it in science, not a math journal because he was embarrassed that math friends would see this very similar thing he did. That's sort of the selfish view of science. Let me just indulge. It's not necessarily a bad thing because brilliant things have come out. Right Newton indulging and coming up with Optics and the calculus and the universal law of gravitation in his spare time. People who've indulged have done great things, but there's another side science and it's science that recognizes the surface to others and the immense responsibility we have not just to get our protocols, IRV approved for ethanol standards. But to accept stewardship of privilege we given funds b, which all come from taxpayers like my mom and your mom and dad and so on, who pay their taxes, not so we could be famous, so we co sit around and scratch our heads, but so that among other things, we can make contributions to the betterment of humanity. And we see that in nutrition science for a long time, the Spirit of Florence Nightingale, who many people think of as a nurse. But in fact, was also a nutrition scientist who recognized that soldiers died more often for infection and hypothermia and malnutrition than they did from bullets and bombs. And she worked on that and she became the first woman member of the royal Statistical Society. She was a statistician and theologist and nutrition scientist advocated for good nutrition for military nursing. It was Widowson and McCants, a very famous scientific team who nearly killed themselves in the early 20th century, experimenting self experimenting with ingested food documents. To protect again the military and the general public so that they could prepare the healthy diets for the UK population during work. We could go on. There are many who have seen that fall to duty. At 20 years ago, little after 911, who was a young man in Connecticut, and he was working in his father's restaurant and learning how to run things. As any of you who have ever been in or seen or worked in a restaurant? No. There's a lot of moving parts, and there's a lot of knowing how to make things happen in a restaurant. Then something else happened in the world. That was 911. That young man felt called That young man who had been to college in appreciable way at that point, said it's time to go. It's time to make a difference in the world, and that young man went to college, go, from there went on to graduate school. And from there joined the Native Army Resource Center as the director of programs. That young man was doctor Stephane Asia. And he worked there for years and understood operations and his restaurant experience helped him understand how to get state and done. And he understood the intense Responsibility at a still a very young person with a budget bigger than most of us will ever oversee. That budget had to be used well for the protection and service of our military personnel and protect would serve us. From there, he said I'm going to protect American Public further and advanced science further, Joint NIH as the new director of the Office of Dietary Supplements when doctor Paul Cokes, who many of you know, because he's on our Agile faculty, retired from acquisition, doctor Pasi He is known as a great scholar himself in the area of protein energy requirements, amino acid, metabolism and effects, muscle injury, prevention, rehabilitation. And we are very lucky to have rescue. Analysis. You do that every morning. I think my days would be more motivated and that was outstanding. Thank you so much. Really thank you for the opportunity to come out today. Share a little bit about what we're doing at the Opposite Dietary Supplements and take you give you a story, a story that the analys already shared about why this topic is important. Why I chose the route I chose, and how did I end up here and why is rigor reproducability and transparency so important, I believe in the field of dietary supplements. I hope you do that today over the next 45 minutes or so. So we'll go over a little bit more about my introduction and overview of the office. Because it's important for many of you, if you're not aware of what the office of Dietary Supplements does, I want to show you that today. But more importantly, I want to show where we're going. O the last year, we've put all of our time and effort in redesigning and reorganizing the office to be more flexible, adaptable, more agile to be able to respond and urgent demands. I want to spend a little bit of time about focusing on what the office has done in terms of product integrity. That's one small piece of rigor reproducibility and transparency. Then shift ears a little bit and talk about the current state of trust and the limitations associated with dietary supplement science. All of that to say that our office is now putting together a new initiative called reproducibility and integrity guidance, optimize research for dietary supplements or more athlete called rigor. That's a new initiative that we're getting up and offering. So you heard a little bit about this. My path is very similar to many of you. Yes, I went through, undergrad, master's degree, PhD. But I did so after years of running small businesses, and I understood the importance of accountability. Understood the importance of setting the example. When I got out of school, I took those principals with me and joined the army as a medical service officer in the US Army Medical De They assigned me to the US Army Research Institute, Environmental medicine, and if they didn't, I had no idea what I was going to do. I draw my eggs in one basket the user because I was dead set on using the skills that I acquired during graduate school to apply those to address questions specific. During that time, I've had the opportunity to run studies all over the world. A variety of military installations here in the US overseas with our NATO partners on top of mountains in cold weather operations, where you have no access to anything but so. It was a very good experience where I've got to take what I learned in the laboratory and apply it all over the place to try to answer important questions for the service. So my research really focused on protein requirements. Specifically, protein and amino acid requirements to support recovery and minimize muscle mass loss in periods of extreme stress and in the environments in which our service members are operating. They typically don't operate in a gym. They don't operate with plenty of food around them. They operate in austere locations with very limited access to food, and those conditions can precipitate seminal muscle loss and also physical and functional decline. And we don't want that to happen. We have this term in the military sure your husband is aware of readiness. A soldier needs to be ready to perform at any point. And when they can't, that's a big problem to the army. Our job as military nutrition scientists is to use nutrition and what we know about metabolism and molecular biology, to develop interventions that offset those declines. A lot of the evidence that we generated over the next 15 years led to policy updates and also combat rational development. So in 2022, I was a chief of the military and Performance Division. I kind of switched gears. I was leading a division of about 90 individuals, about 30 PIs, whose mission it was to design Appropriate and accurate and valid interventions to mitigate musculo skunal injury risk in soldiers. And one of the biggest issues in the military is soldiers constantly become injured. The trip from the army right away from basic combat training that has a loss in terms of finances and also a loss of personnel, and they become injured more and more and more over time. You can see the snowball effect that might have on an individual's career and the subsequent health care costs that places on the VA, and the external system wants to get out of service. But in May of 2022, I got an e mail from the search committee chair for the Office of Dietary Supplements Director position. And I was kind of intrigued. You know, I am a nutrition scientist, dietary supplements wasn't necessarily my spin, but I did work in the context of dietary supplements in the form of amino acids. And I saw this as an opportunity. What really bothered me about my field is that no matter what you say in your paper, no matter what you say in a presentation, and get misconstrued and used as saying that I am all for this or I am all against this, and this is going to cure everything you have or this is snake oil it's not going to work whatsoever. I felt that this was a unique opportunity to come in, bring a level of objectivity to the office dietary supplements and really hold in on key issues of importance to public health and drive objective and strong science forward. So after a year or so, I got the offer and lo and behold, I'm here. A lot of change has happened over the office since I've been on board, and I'll share a little bit about that today. So the history of the HF and dietary supplements, it was formed in response to a law. So in 1994, there was a Dietary Supplement Health and education Act. That act was an amendment to the Food Cosmetic Act, which really said that dietary supplements are now going to be regulated as food. Now how that actually happens is that the FDA is associated with the regulatory policies of dietary supplements. They do have certain forms of regulation. They certainly don't have the regulations associated with pharmaceuticals, but they are regulated in the context of food and food ingredients. We on the other hand, the partner office to FDA's Office of Dietary Supplements at NIH, our job is to promote dietary supplement science and do so making sure science that is being done is of the up utmost quality so that the public can have credible evidence that they can make the right informed decisions about dietary supplement use. NIH is huge. I don't know how many people if you have seen the campus. Is the college campus in and of itself. There are 27 institutes and centers. And then there are a bunch of offices within the office of the director, and that's where the office of dietary supplement is fixed. So I fall under the immediate office of the director in the division of Program coordination Planning and Strategic Initiatives. That's a now Eckes is what they call that. Okay. I don't know if this wasn't in front of me that I could actually say what the division is called without it being right in front of you. So Nila say they are working on a name change for that. The only problem was that is that it has to go through Congress for any kind of change for that. There are a number of offices within DPC. They all have the same job. Our job is to really coordinate science across the ICs, get the ICs engage in research, stimulate innovative research, pull people together, meaning resources and expertise, and identify what those critical needs are in science and put out the funding initiatives, manage large scale programs to really move things forward and make a change. We are one office within that program. What do we do? So the mission of the office of Dietary Supplements is the coordinated collaboration of innovative and cutting edge dietary supplement research across NIH ICOs centers and office office VNO A other federal agencies to foster knowledge and optimize health across the lifespan. We do that a variety of different ways. Our primary way by which we do that is we support research, use a variety of df fund Unlike an IC NDK or National Institutes of Aging. The DGs offices do not have direct funding authority. We use a variety of other mechanisms actually fund work. So if you were to submit a fund grant application for an RO one to NIDDK that had a dietary supplement component in it, and they score it very well and they were going to fund it. They would come to us and look to us to help co fund that grant over the life of the grant. That's how the DPGpzi offices help support science in the true funding of grants. That's one of one Other ways we do that is through contracts, government contracts, through interagency agreements, and also by coordinating large scale programatic efforts, where a lot of ICs should have been money to really go after a big effort together where you can get much more or bigger bang out of your buck when you have much more in the bank than you do with one DCC office alone. That's what we try to We also prioritize trying to advance the workforce. We have a number of different training opportunities for individuals interested in learning about dietary supplement science? We have a research practice that goes over a course of two days. I basically takes you from sim to nuts on how dietary supplement science actually occurs from all the way from product formulation, the analysis of that product, to running the studies, all the way out to the market in the trade organizations lobbying for dietary supplements. How does that whole piece work together? One of the most important parts of our office is communication, people take dietary supplements. Some of the estimates are on average in foster population around 58% of the population will use at least one dietary supplement. Of that, a significant amount is obviously multi vitamins and mineral supplements, but there's much more. As you increase in age and between sexes, those rates increase dramatically and individuals over 70 n and women report using about 70 or 80% of men and women report using dietary supplements. So nearly all older adults report using some form of dietary supplement, and the underlying reason that they take it is to improve their health. And our job is to understand whether or not it actually does that at all. So when I got there, I sat down in my boss's office. This is 2 hours after getting my ID. Sat down with his deputy director and he goes, we're so happy to have you here. We understand your last job. Put a lot on your plate. They asked you to make a lot of changes. Well, we know you can do it, and we want you to do it. I said, I just walked in the door. So what are you looking for? We want you to bring ODS package be fold and make it a powerhouse here at NIH. There is so much work. I'm going to show you in a subsequent slide. So much work being done in the area of dietary supplement. This is not enough coordination. We can do better if we work collaboratively, and that's one of the main things I've always did when I read my own. The first thing I had to do was develop a strategic plan. That was the bulk of my first year. That strategic plan is now online. It has about two more days left in public comments. If you go onto our website, there's a link for the strategic plan that's in draft form. Once we get all the comments back, that will be revised and it will be out published and it will be our active strategic plan by which we will operate for the next five years. Let me just give you a big picture overview of what we're trying to do here. I wanted our office to be fundamentally based on scientific priorities. We are a very small office with a medium budget. We can't be doing everything and we can't have N of one silos doing research. We need to come together as a group and identify what those priorities are and put our heads together and get some work done. Identify three broad categories and those are diverse population, healthy lifespan and resilience as our scientific priorities. Those priorities build upon the accomplishments of the office also emphasize our ability to pull in ICs who do research in these areas because they are broad enough. From there, I needed to build the foundation by which we operate. That really is the academic disciplines that we hold as strength within the office and the individuals that we collaborate as biological sciences, population sciences, and analytical sciences. And from there, I worked out from there, building the infrastructure. What infrastructure did we need to make our office actually function well. But we needed to advance methodology needed to apply all the skills we had, and we needed to reach out and branch out and figure out new ways to make the feel stronger, we needed to be really strategic on how we used our funds. We don't have money like National Master Institute. We really don't. We have a very small pocket of money, so we need to smart and strategic and how we invest that money. From there, we have our operations programs. Who keeps the office moving and running on a day to day basis? And that's all the programs on the outside. The key tenets to the strategic plan, I just want to call out are those highlighted, more trans IH, proactive versus reactive posture. Rather than wait for someone to come to the office with an idea for dietary supplement. I want to go out to the ICs and say, this is what I think is an emerging need that needs to be addressed. I have done that so far with several ICs, and we're starting to generate interest and some shared interests that I think are going to go forward into new initiatives. Regular reproducibility and transparency, that was one of the things I parked on during my interview process. Obviously, it's here in a strategic plan. The one thing that really I could never do if I was a leader is maintain status. Now, I'm not a manager. I know I have to manage and manage people. But my goal for the office is to provide vision, provide that big picture vision, provide those high level areas and motivate bide opportunities for my staff to go after those and be creative and curious to really address those the best if they can. I won't be keeping things as it is. That's really not my style. So going in there is about shaking things up a little bit, about really getting the office to the next level of prominence. So just a little bit more information about our cross spending priorities. For diverse populations, this is pretty straightforward. I understand the use of dietary supplements and their effects on health requires investigation into their use and impact in diverse population in the various intersection of these groups within the United States. Lifespan so should be pretty straightforward too to understand the association between dietary supplement use, disease prevention, and health optimization at different ages and life stages in the biological mechanisms through which dietary supplements impact health. And resilience, was kind of a budding area when I first got into the office. Vern Brown led a trans NA workshop to really define resilience for NIH, to define a research framework for resilience, resilience to the body's ability to respond to stress. So they led that initiative before I got on board. And when I got on board, I said, you know what Vern, I managed the portfolio on resilience in the EOD. I'm all for this. I understand that needs a heck of a lot of work for a DP, a more prominent area in the field. But a lot of people are using the term. I'm not sure if they use it all right, but a lot of people use the term. Let's do something and really make this intent. Resilience is one of our three pies. I want to quickly highlight a couple of emerging areas that we are going to act on. There are a good reason why we're doing that. Asesimated that about ten women will turn to dietary supplements to help offset the sichematology of going through medics. The efficacy of using those dietary supplements is unclear. Also how they use those dietary supplements, the exposure, the doses. We don't know very much about how dietary supplements are used in this population. We don't know how they're used across the disparate population in the US. You may be aware that a year ago, the president signed the Women's initiative Women's Health initiative, and that is being led out of the office of Research on Women's Health. We are working hand in hand with them to develop new scientific initiatives that look for non pharmacological treatments to help women at different life stages and menopauss being one of them. We are in the beginning stages of this. They just got their budget execution for FY 25, and we're working towards developing a workshop, and then the next steps will be what the funding requirements will be or funding announcements might be that come in. Now, one that's kind of near and dear to me, not because I'm a big GLP one receptor agonist person is because the misrepresentation of sinus. At ASN past year, American Sci nutrition Demos and chaired a session that was specifically on GLP one receptor agonist. It was amazing how many sessions at that meeting focused on this medication and focus on a variety of gals. I'm not interested in GLP one for its ability to improve insulin sensitivity or its ability to induce weight loss. I'm interested because the dietary supplement market is already decades past where the field is in terms of understanding how these drugs work in different populations. And this is a picture here from GNC. They already have all walls of dietary supplements that are meant. For GOP one. You're dealing with GOP one side effects, we got cure, and here they all are. They're just repurposed dietary supplements, I would assume that are just being said, well, this can help with nausea, this can help with constation, and got help. If you're losing muscle mac, we don't even know if that's legitimate at this point. But if you're losing muscle mass, here's some amino acid and protein supplementation take on top of that. What I want to do and where I see the gap is that the potential effects on nutritional status are one. If you have a poor diet to begin with and you're on this drug and it is changing dietary behavior, you don't want to eat. Change vegetables all day long, that nutritional status can be compromised. Access to healthy foods in that population could also be a issue. Couldn't change physical activity couldn't change b morphology, fracture risk, muscle quality in a population where musculoskeletal development is so important. I'm really looking more towards adolescent and young adults as a starting point to get moving. That's fit this figure here just shows a number of publications on GLP one receptor agonis as a whole. That's a dark line on top, that basically just says that there are a lot of dietie, a lot of studies on GP receptor agon. You look at the bottom of that point here. So if you look at the bottom of the graph, that is studies on just adolescents and children. I hasn't moved. So all of the studies that been on adults right now. And the reason why that's important is because the prescription rates on GOP one receptoragonous drugs in adolescents is sci. So this paper was recently published in JAMA in 2024, and they looked at prescription rates in individuals 17 and 18 to 25. They just looked at the prescription rates 2020-2023. Increase, dramatic increase in rates. Yet all the studies have been done on s. So we don't really have a lot of information, and likewise, we don't have a lot of information on any of these dietary supplements in this context. So I'm trying to generate some interest to see where this goes. I'm not looking to resurect the idea of weight loss supplements, but I want to know where this is a public health issue of concern. There are a lot of things that we don't understand about at this point from a nutritional standpoint. Just shifting gears, I'm going to talk about ODS and its mission to advance research integrity. We have three efforts to do that. One is the analytical methods of reference materials program or ARM. That's a program that's been around for about 22 years at this point. We have the NIH consortium for advancing research on mechanicals or other natural products or carbon. That really stepped from the NIH mechanical research centers back in 1999 and morphed over the last 25 years. Within our purview of reviewing grants, we also have a strong section on product integrity and we hold grants accountable to certain things. That's all summed up well in this figure here to the right, where the goal here all the way to the left, all the way to the right is you want to understand the phenotypical biological effect of the dietary supplement. If you work all the way backwards, you can't do that and you don't know what's in the dietary supplement. From a natural product standpoint, they spent we spent a lot of time trying to understand what are the constituents of that natural product? What are the active ingredients, where are the acbive ingredients, are they interfering with anything? What is actually accessible in the body? And then subsequently, I will available, would actually have an inaction on a various pathway in this case, g translation. So Those are all the steps by which we try to look at in the context of product integrity. Do we know that? Because if you're running a randomized trial and you have no idea something does, and at the end, you have an finding, what did you do? So that's an issue that has been a concern for the field for quite some time. We're working on that and we'll continue to do so. Just a little bit more information about ARM, it's focus on rigor and reproducibility. The goal of RMR is to provide materials, reference materials is a close collaboration with the National Institute of Standard and Technology in Gatsburg. And it works to coordinate and develop and disseminate resources that promote accurate and reliable chemical characterizations of dietary supplements, natural products or clinical biome. Just an example of some of the things that we produced, 56 newly available or reissue dietary supplement ingredient reference materials, and those are available on our website or available on this. They're also publicly available for I think Fisher, biaical, self. Ten newly available biomarker reference materials, and these have been referenced are cited quite a bit in the purview literature. There are people using our products. We also have other things available to researchers. And that is we have a data resource to facilitate dietary supplement characterization. Everything that we have done over the years is available on our website that a searchable that can provide people the right information to go and buy the materials they need to assess their dietary supplement accurately. A big win, I would say for AMA is vitamin D and standardization of vitamin D measure. The AAM program in the context of vitamin D developed the reference measurement procedures that are accepted by the Joint Committee on traceability and medicine for vitamin D metabolites and serum. They also developed the vitamin D binding protein, measurement and serum and developed the method for determination of vitamin D and 25 hydroxin D three in food and supplement agencies. So that kind spurred on the offices vitamin D initiative, and subsequently the vitamin D standardization program, which really came up with a consensus guidelines to standardize clinical measurements of vitamin D. And develop a methods for retrospective standardization of vitamin D status collected in clinical and population studies. This is a big deal that really helped to clean up the vitamin D data that was coming out probably 15 years ago when vitamin D was in craves. I would say the cure all for every issue you had with vitamD back in the 2010 period. So our work here really helped to standardize and make that matter. I'm not expecting anyone to read this, but what I wanted to show you here is that if you apply for a grain and it comes our way, we do have a lot of questions and answers available on our website about product integrity and what we would be looking you know, and this gets to the question, well, other ICs do the same thing when it comes to their grant applications. Do they describe this well enough research can make sure that they are addressing the appropriate things that we or the funding body wants to know are being done to make sure that the study is strong. I just want to show that here to give you one indication of what we are doing. But altogether, if you heard we say, all I've talked about is product integrity. That is one piece of the puzzle in terms of dietary supplement science. I have not talked about once experimental design, statistics and reporting. I challenge the staff when I got it and I go, let's look bigger. We'll continue to do Amandi that's important is making benefits. But let's expand our efforts to go to the complete picture of the experimental designs. Let's try to move away from a stetary supplement study of an N of five that's not randomized. Let's try to move away from that and make it more important and more impact. So I'm going to shift gears here to trust and dietary supplements sites. And this is the paper that just came out this came out last year. Health claims on doses of fish oil supplements in the United States I was published in Jama cardiology a brief report. The graph represents the variety of health claims. So they use our dietary supplement label database that we house. They looked at 20 100 or so fish oil supplements, and they found that 70% made at least one health claim. Those are all reflected in the bottom of this wrap. There's brain, joint, I, antioxid, cell membrane activity. The various categories that those health claims were nd 81% of those claims were structured function, meaning not on a clinical health outcome. 19% were actually able to use a MDA quality assured health claim. So only 19% of those actually met the criteria for an FDA claim. So that's significant. And then the doses that were used in these studies were all over the place. In some cases, the doses in the studies that had the labels that said that they met the criteria for a qualified health claim did not have the doses that actually would achieve that qualified health. So that's problematic. And I just on the right hand side, just show you what the actual qualified health claim for fish oil supplements are. And you can see even the FDA does not put out a conclusive statement. They're not saying that this is going to fix ADN C, but it is giving you something that's more than saying supports phone, protects muscle, that would be a structure function, a claim based on a mechanistic study. Then there's the impact of science flotation. I found this term to be pretty interesting. I find myself being someone who actually probably does it more than I should. It was coined by Timothy who's a lawyer, it describes how brands and markets borrow language from emerging areas of science to market improvement products. The buzzwords are like supports, boots promotes terms that are not quantifiable, but researchers media and marketing lead into these to really get vague health cleans out there and to stimulate people's interests to a product. There is an interesting paper in New story in the New York Times, how fake science sells wellness. It's true. You see here, research fact, medical grade, evidence fas. I can't tell you how many times that when I've done like an elevator pitch to a general that I say, Oh, what we're doing supports muscle. Well, that doesn't mean anything. But now I see that this is a thing that people when they just take buzzwords that we use just use without even thinking about it as scientists and those get pushed as having a meaningful health. A year ago before I started the office hosted doctor Marines. I don't know if any know she is, but she's very well known in the nutrition science field. And she gave a pretty provocative talk to the office. At one point she challenged the office. Her whole thing was about the politics of food science and the issues surrounding dietary subtle. And in her presentation and it's available on her website if anyone's interested to listen to it. She says, who's causing mistrust? She said, your office is contributed to mistrust in the field of dietary subtle where we sit back and say, well, I thought we were trying to do the opposite. Well, she says on your website, you'll have things that say everybody takes them, but not everybody needs them. Some have side effects, little evidence or benefit, need more studies, use precaution, FDA doesn't So I heard that and I said, well I'm not going to agree with everything she says, but I understand her point that the office who's trying to advocate for credibility is providing contradictory statements within the same document. So I think we need to do a better job of understanding how we message science around dietary supplements. So I just thought this was important to share. Like I said, I'm all for all of her points because it was a pretty pointed presentation that got a lot of feedback. But anyone wants to listen to it, it's available on. This table here was recreated from a recent manuscript published in 2021, and it talks about the limitations in nutrition science. Well, there are a lot of nutrition scientists in here who have clinical nutrition studies. There are many many, many, many problems or issues or limitations in terms of ni nutrition science. And I would say there are so many limitations across the board unless you're dealing with a potent drug trial as regulated by the FDA. So when we get into the nutrition context, I'll just poll out a couple of big ones. Complex nature of dietary trials to bear with conventional pharmaceutical trials. Okay. Relatively low incompatibility of dietary clinical trials with good clinical practice guidelines. Most people want to avoid that because it as an extra layer of regulation. For experimental design for adaptation with denational designs and put in trials, a slew of methodological problems and weaknesses. For reporting of trials in scientific literature, limited translatability of the observed effect, due to the effect size and the herogenita of the data, and high susceptibility to the founding variables. All of these limitations are there for every dietary supplement or dietary trial, including dietary supplement trials. More importantly, when you get into dietary supplements, there are more limitations that are above and beyond what's simply involved if you want to do a controlled feeding study, which I was saying much better than most. For example, we already talked about on the left hand side. You need to know it's actually in the dietary supplement. But once you know what's in the dietary supplement, doesn't mean that when it gets to be gut, it's actually dissolved and it disintegrates and it's bio accessible. There are studies that you can do and they are expensive, but they are disintegration and dissolution tests to understand of that capsule or that matrix, whatever that gummy, what is actually available and accessible to be bio available in tissue. Well, that those tests can be done. The problem is they're very expensive, they're labor intensive and not everybody has access to. But these are very important because you could very well have all the best intentions in mind. You know what your active ingredient is, but maybe only a tenth of it gets to where it needs to be. That's a problem. I think there's been some funny jokes or names about dietary supple that just sit in your gut. I can't really remember the needs, but Johanna Tier one of our consultants says this every time we talk about this issue she send just sits in your gut like a rot and doesn't do anything. I was like, Oh, That's that's what we're investing in, but I understand the point. These are just some of the steps that take. Then you understand actually have this modal available, the uptake and response in the cells, and then getting all the way over to maybe first rodent model and eventually a human. These are the steps. How many people bypass this and go right to here go right to animal or human trials, a lot of people. I'm not saying every study needs to start from the left all the way down to we'll get back to the left, but these things need to be considered to do a well controlled dietary supplement trial. You need to understand these variety of factors. The ter turn on that, anything is flooded with dietary supplement trials that are filled with limitations. We just pulled a couple of systematic reviews that were published this past year by one group. In one publication, they looked at 438 eligible studies, but they excluded 308 of those studies because of no randomization, no lack of relevant clinical outcome, short intervention periods, and the inclusion of participants with severe disorders which would have compounded. And they found very similar things when they looked at other micronutrient supplementation trials, and this are just another couple of references. The problem is that all of those studies, just because they didn't get involved in a meta or systematic review, they still exist. If you get these one offs, may not be the strongest study in the world, may have bias within the studies. They're still out there, they're still in the literature. People still refer to them and they attribute to the noise in the field, and they contribute to the confidence issue that we have in dietary supplements science. Well, this is not something new, it's been recognized from a lot of people that we have to do something to make nutrition science better, and with that comes dietary supplements. This is just a variety of recent review papers or perspective pieces that really talk about how can we make the field stronger? Well, papers are good. Fact, this paper here is Paul Coates, this is blue here. This paper here, I saw I say, look, even Paul is still thinking about this. People are thinking about ways to make nutrition science better and better reporting, better scientific design. But who is the one that can actually make a difference and try to do something about it? Well, considering we find the majority of dietary supplement sciences in the US, we should take the lead and try to do something to make you feel stronger. So should we or do we have an obligation? I I would question if we have an obligation. I'll tell you why. This is the funding layout for 2004 FI 22. Feral research funding and FI 22 is 32. Dietary supplement related funding was 241 million in FY 22, and that means anything related to dietary supplements. It doesn't have to mean take a supplement and measure this two months later. It could be mechoistic work or anything like that. Direct dietary supplement NIH funding nearly 71 million in FY 22. We contributed to 7.8 million that. In the grand scheme of things, there is a lot of money being spent on dietary supplement science, whether direct or indirect that is being funded by NIH. And could we be doing a better job of holding scientists accountable to integrity? And Rigor, we probably can. Now, this is just I'll provide the slide of David afterwards, in case you guys look at this. We ran a portfolio analysis. One of the tools that we have at NIH to really look at what's been done at NIH over the course of the years. So this portfolio analysis is a very quick and dirty analysis of FY 22. I asked doctor Kuzak, who's going to lead the Rigor program. Tell me whether or not all of these grants had information about rigor and reproducibility or product integrity in their grant. Well, on the right hand side, there are a variety of different statements that were included. Some of them just say they allude to a authentication of sources. They allude to source overall strategy and methodology. They don't give specificity at all. Some go into a great deal in terms of this probiotic study at yet. However, this is only a few rows of projects and this page eight, this one on multiple pages, and the majority of them were empty. There are a huge number of funding announcements without rigor in some form mentioned. There appears to be case by case driven by the IC, without any standard guidance. What can we do? Is NIH do this already? Well, they do. There is a great need for rigor reducibility of transparency that's well recognized at NIH. At the National Institutes of Neurological Disorders and Stroke, they have their Office of Research Quality, whose mission is exactly that to promote experimental analytical rigor, measures to reduce bias, transparent reporting, high quality scientific research. Well, they're working on it. There are efforts like this at NIH, that's a lot more than just NINDS. For example, the Office of Disease Prevention as the improved preventive research methods initiative. That's one of their key priorities that they make sure they focus on every year is evaluating the state of the science and looking for strengths and weaknesses and be able to provide up to dated information pertaining to prevention science and how to make it better. For example, they serve as a resource to ICOs on prevention science methods as they develop new funding opportunities, workshops meetings, and other activities. They collaborate with the ICOs and strengthen their policies and procedures to encourage applicants or investigator and shape awards to use the best available methods and application for prevention research. They review and conduct research methods, use in prevention research, using the portfolio analysis technique. This has made a real difference in the prevention side effect of prevention research at NIH. There are models that we can follow. When I got in there I said, well, we don't need to recreate the wheel, we need to learn from our colleagues at NIH, and we need to build a system in place that can actually do this at the office of dietary supplements. So that leads to our Rigor program and that's being led by doctor Adam usa who is currently leading the analytical methods of reference materials program. Adam was the perfect choice. I have never met a perhaps more detail oriented critical scientist than him I asked him for one thing. For example, I asked him for the portfolio analysis. Ten pages of funding announcements and all that little text. He's very good at doing this, and I think he's going to be a great me for this project. Let me give you a little idea of what this is going to be and where we're going to go. Well, the aim is to continue and grow our efforts to improve the rigor and translatability of dietary supplement science and related nutrition research. By really going after two main questions. Well, how can we as an office promote more comparable and reproducible evidence based on mechanistic and clinical research, and where should NIH resources and money for dietary supplement analytical sciences and our methodology capacity be prioritized? Those two questions really are the starting point for our conversation. The big picture goals. We want to expand our tools and collaborative efforts to promote progress in the rigor in which dietary supplement studies are conducted. Designed, evaluated for NIH funding consideration and reporting in literature. In my mind when I talk to Adam, east three are the most important. How studies are designed conducted, how they are evaluated and ultimately funded, and ultimately when they get to a journal, how are they reported? This is where I think we need to really focus. There's obviously a lot of details in between these. But if we can do this or even a portion of this, we will obtain a difference in. Initial steps, which Adam is doing right now, conducting NIH portfolio analysis, assessments of where NIH is addressing methodological challenges, supporting high priority research, and where more is needed. Etransion and development of resources, the age research, and experimental design, conduct, and assessment reporting and elevating dietary supplement research at NIH. The reason why I mentioned Elevate is because there are a lot of groups at NIH working elements. We need to have a set the table, and now because of Adam and because of this effort, we are now elevating dietary supplement to that conversation, so we'll learn from that. The anticipated products are just listed right here. Our goal is basically to try to make the field better provide guidance to funding announcements, provide guidance for researchers to help reporting on the back end overall, lead to more credible guides or supplements. So with that because of all that, I just want to thank you so much for your attention. If I can hear your questions, I'll answer them, but again, send and everybody who help this f happen. Thank you very much. I appreciate it. Okay. Happy question. Thank you. The wonderful presentation. Question have you got a huge incumbent lucrative business of lots and lots and lots of people selling lots of products through lots of channels, and as you indicated, very little rigorous science back in the mop. How do you handle the politics? How you in terms of moving through your vision of where it needs to be versus the starting state here. Can you talk about that and certainly doctor Senator Hatch from Utah, is meddling at all of this in terms of making your job infinitely more difficult. Yeah, the politics is different. As a DOD scientist, 20 miles out of Boston, you really didn't get into that. But it is present in this job. The biggest link is between our office of legislative policy Analysis and the trade organization. So there are a number of trade organizations that represent primary dietary supplement companies. Yeah, the Mom and pop things, I don't know what you're going to do about that. But they are the ones lobbying Congress for what needs to be done, where the emphasis should be. In my meetings with them so far, I said, My job is I'm going to come right down the middle. I'm not going into this with any bias. My goal is not to be gotcha. My goal is to be fair, will actually be a good sign. The relationship with them has been really great so far. They felt like the office and many have been in this gotcha. Look, we got you, it's bad. It's not doing anything. Now, what I said to them I go, I'm going to call out science when it says they do nothing. But if there's a study that says it works? I'm not going to be afraid to say that study worked. I'm always going to go, I don't know, it may work, it may not work. The politics is tricky because it comes from these big trade organizations and so far with Congress, the most that they have been influenced so far is the trade organizations some of them have been pushing for mandatory product listing with the FDA. We have almost the exact same thing. Now it's going to be the question of, does that actually get enacted into law? Will FDA establish this database, and what do we do with ours? There's some politics of that that will have to get a. S. It's fun. It's new it's different. They don't teach you that at school. Thank you. From an analytical assessment perspective, how far do you think you go? In the sense of there are organizations that publish official methods like USD, AAC, and in many cases, they'll have official methods for some of these bioactive substances. In your case, you deal with academic community a lot and my experience academic community doesn't necessarily care about those offical methods. My experiences are to try bios whatever excuses are too hard, charity practice, whatever. Would you think about actually recommending these approved methods and say here one, if you're going to work in this space, whether it's re clinical or clinical? Because to me that is foundational. It's number one, that one slide you have, you really need to know what your substance is, and how much of it you have. I mean, absolutely. I asked at dinner last night I was kind of giving them Some questions I gave myself when I first started and one of them was how you tell whether or not your program was having a benefit to return on investment. For that, any Methods program, I said, do we require people to use our methods? Do we have some criteria by which if they're asking for our funds and they're studying whatever Kirkman or something, that they're using a validated method that we know is valid or they're using something from AOEC or USP that we let know it's real versus just method developing in your own lab and trying to figure out We haven't yet, and I think that's a limitation. I do see as we move towards this, the guidance will be, if there is a validated measurement or reference material, you must use that because this way, we know that which studying is what's it going to be as opposed to shooting in the dark and hoping to got. And the follow up question. Then to the jury s. Yeah. Because often I would say when an article goes to a nutrition journal. The reviewers probably aren't focused on the analytical nature of it. So how do you get that translated from that first part to then the journal part where you have reviewers that know k. I'm looking at the analytical section, and you know, I I made this connection to OPS. And how they analyze this is it's like tier three for them. So we're going to start asking a lot of questions. Well, from the journal side, I think that's going to be the heaviest left of all of but there is There is evidence of some doing that. Connie Weaver led a group to try to amend the consort diagram for nutrition RCTs. That was published in Advances last January. I saw that. I was like, Well, maybe maybe we can do that. I a lot of the journals involved. Reviews are only as good as the actual reviewers. That's the limitation. So clear instructions. I don't know if everything will be an amendment to a consort. Maybe maybe you know how journals have expertise in statistics, maybe expertise in analytical chemistry for various journals. I don't know. That's way down the road, you know, in terms of the back end of the journals, but at least I know Connie and others are coming to give a seminar to the office early fall, talk about how they went about doing that for their effort to amend the consort diagram from nutrition studies because I think we have to do something similar. People need to you need to be you should be reporting exactly what you measured and that should be in the paper. If someone is someone remembered the whole thing about papers, you should be able to take that paper and redo the study and get the same thing. I don't know if that happens. So that's just we'll see what goes on that and I don't know I don't know what that will look like. Thank you. Great to. Thank you. Thank you Sha you mentioned FDA does not have regulatory someone they don't regulate, but you have forse au cross over into the drugs? They do regulate them in the context of foods. Right. Okay. But if you cross over into the drug space. Oh that Differ my question is does your agency have any collaboration you're seeing literature and see the trigger words. Do you have any enforcement requirements for the FPA partnership? Well we've worked closely with that group, but nothing in that sense. And a couple of words that stood out from your question. The word treat the sun from to play that actually is called out as prevent optimal but not treat a known disease. So that usually has prevented us from even playing in that space, but we'll see where things go. So I don't have a faith. I wouldn't say my goal is for the best evidence. I don't ever really design even, right? If that's what you observed or you observe it epideolgically, then, you made that claim, but Most of the times you're looking for a biological post. I don't I don't have one that I want to prove. I just want the science to be given. So I would be that amino acid. I'm like waiting for everybody to show me that amino acids ma you speak the strongest guy in the world. You know, I'm not looking whatever the evidence shows. So the office has being, I would say conservatively towards preventing harm, and you still promote that when there's reason to promote that. It's not there for everything. So if I need to make that message or say that message first think I will. But I don't have a I don't have one area of applicant. I really hope that let's kick ins. Station question was related to product integrity or authentication. So a pre clinical model rely on lab or collaboratory work with use of the chemistry. But if you're doing a trial, humans usually providing a supplement formula. So a third party authentication be recommended or would you recommend on compound interest like nitrate. You can do that. The issue is with a lot of supplements aren't good biomarkers that are reflective of the supplement doing anything. So there's an issue on that. Lack of biomarkers in nutrition in general isn't the greatest aspect of our field. If you're going to buy something off the market off the shelf, it says USP certified or something like that, When I have done that, I've always sent them out to I don't know what you call it used to be called CBNS. I would send them out to an external laboratory to analyze. I was always dealing with protein amino acids. I was a little bit easier than a natural product, but there are labs that do what I showed, which will I would work that into a budget, send a lie of what you have or several samples from a lot of supplements. I just one validation. For example, when I would publish on amino acids, I would say, this is what I wanted in it. This is what was analyzed, this is what they got, like a supplementary table or something. So I can at least report like exactly what within supplement. So I mean, if there are labs that can do it, I don't know off the top of my head, know all of them, but I could look into that. Hello. Thanks for tok. This thing cares a lot about transparency and about reproducibility. Where are we standing regarding the data availability stories? And what can we expect from pis in the next couple of years? I would say you can expect some calls for proposals that would go after reproducibility, taking existing data and trying to reproduce it. I you would expect something coming out to help us build the case for making things better. There aren't a lot of there's a growing body of I would say momentum at NIH now for Reproducing studies. Common fund, for example. They are now going to be forced to ask people to reproduce and replicate findings. So if you are funded by the C, I don't know if everyone knows what the common fund is, but big money studies, hundreds of millions of dollars studies and like motor paths people want to know if those are real. So now they're being asked by Congress to make sure there that you can repeat them. You get the same data. There is a bigger push for my office. That's a big scale. I can't afford that, but there are smaller things that we can do. Like some of the papers you worked on, I think that's a start, you know, in trying to understand what's been done and, you know, are things being done a report subpopulations ogenity in the patient, comorbidities and gender, et cetera. To what extent understanding the impacts, differential impacts in terms of different sub populations of these these actives that you're working with or supplements, any movement there or Yeah. Yeah. I was at a David was at the nutrition meeting in the fall. And one of the big things they talked about was diverse population, populations of health disparities, and I mean that's why it's one of our priorities. I want to understand how these supplements are acting within various groups. I want to understand if the heterogenic response is dictated by some demographic or phenotypic aspect that we don't know already. I think that could open the door for a lot of new research. I think there is a way to potentially apply, I would say, nutrition science. Truly understood AI feel if that was the case, starting to put the pieces together to understand you build like a phenotypic signature of how a population responds to a dietary su. That's exploratory work. I would love to see something. Thank you so much. So we have a beautiful and award for doctor Pasopos. And it's a glass award, and I will read the inscription. It says, This is a quotation. Almost every major systematic error which has deluded me for thousands of years, relied on practical experience, horoscopes, incantations, les, which The cures of medical practitioners before the advent of modern medicine, were all firmly established through the centuries by these methods. The scientific method in contrast was devised precisely for the purpose of elucidating the nature of things under more carefully controlled conditions and by more rigorous criteria. That is from Michael Pan personal knowledge. Then we have added. Thank you for lighting a candle in the darkness, promotion of rigor in nutrition science. You are following the tradition of Florence Florence Nightingale, who is known as the lady of the lamb, because she brought the lamb into the darkness, the date with a better nutrition.School of Public Health-Bloomington
1025 E. Seventh Street, Suite 111
Bloomington, IN 47405-7109
Phone: 812-855-1561
Fax: 812-855-4983
