So I will do the next presentation, and I will be talking about how guidelines are developed, overview of guideline development. My disclosure, important disclosure here, here are my TOIs, but my important disclosure here is I am not a methodologist, and I knew very little about how guidelines were developed before I started to become involved with this committee. And I've learned a tremendous amount and wanted to share some of the complexities involved with guideline development with you. And so the learning objectives are twofold. One is to recognize the many steps involved in producing a clinical practice guideline. And again, we're talking about clinical practice guidelines, not reviews and not consensus statements or anything else, but clinical practice guidelines. And then identify the various challenges in each step. And as we've gone through various guidelines within the Guidelines and Assessment Committee, I can confirm to you that every step has its challenges, and I'll try to point those out as we go through it. Renat put this definition up just a moment ago, and I will point out a couple of highlights from my perspective that are critical in this definition from the Institute of Medicine from 2011. They talk about advances in medical research have reduced the level of uncertainty in clinical practice. Clinical practice guidelines, CPGs, complement this progress by establishing standards of care backed by strong scientific evidence. CPGs are statements that include recommendations intended to optimize patient care. These statements are informed by a systematic review of evidence and assessment of benefits and cost of alternative care. So the key here are in highlight here, in that they are backed by strong scientific evidence. There's recommendations in clinical practice guidelines, and these recommendations are provided after a comprehensive systematic review of the benefits and the potential costs and harms of those recommendations. So that's sort of where we're starting off. And as I said earlier, there's a number of different types of documents beyond clinical practice guidelines that clearly have value, but their production is a little bit different than clinical practice guidelines. Their expense is different, and honestly their value is different to patients and clinicians. So this will be the overview of what I will talk about in a little bit of detail, but these are the steps involved in the production of a clinical practice guideline. Some of them will take longer than others, but we start off with a guideline proposal. We form a work group to produce a guideline. Disclosure of conflicts of interest is a critical piece of this puzzle, and I'll talk about that in a little more detail. We develop a protocol, and then we get the approval of that protocol, and that has several steps in it as well. We then identify evidence. We screen the literature. We rather search the literature and screen the evidence that we have. Once we identify the evidence that we're gonna use, we have to classify that evidence. We have to formulate our recommendations, write a draft guideline, review and get approval for that final guideline, and then the most important step is implementation, dissemination, and finally, updates. So we'll talk about all of these in some detail, but we'll start off with the guideline proposal. There are two ways guidelines, several ways guidelines can be proposed. One way to look at it is top down or bottom up. In other words, the society itself, the American Epilepsy Society, for example, can decide what is contemporary, what is important, guideline assessment committee, go produce this guideline because we as a society think it's critical. Alternatively, members can approach the society and say we think this is an issue within our practice. Please provide us guidance as to how this ought to be managed. These are the two ways this is done, and of course, there's many permutations. There could be groups of individuals that band together to request this clinical practice guidelines, but those are the two main ways. Once this comes to a guidelines committee such as ours, we will vet the process, we will determine the need for such a process, and determine the bandwidth of our society, of our committee to undertake such a guideline, and then if that is accepted, it is approved, then we find funding for it. Moving on. Next is formation of a work group. This work group will then produce the guideline. The key people in this work group are gonna be the chair or co-chairs. Typically, it'll be more than one chair. Co-chairs are typically needed to keep the guideline moving forward. They will be the point people of this guideline. Typically, a facilitator is very important. This will be an administrative person from the society itself, or from a large group of individuals. One person is sort of the manager of the guideline. Not the co-chairs, but the manager of the guideline, making sure the administrative work is getting done. Content experts are, of course, critically important. People who know the literature, who know the material at hand. Guideline committee members are also relevant. These are people who know how guidelines are structured. So these are people in our committee, and our committee serves to educate these people on guideline development. They will not serve as content experts, but process experts. Patient representatives are critical. You heard Renat talk about the need for patient representatives as addressed by the Institute of Medicine. People who are impacted by these guidelines. Epileptologists working at a high level may not be as directly impacted as general neurologists and primary care practitioners when we think about how guidelines are implemented. Other relevant societies often will have members on this writing committee. A methodologist and a librarian might be essential. Typically, you'll have 10 to 12, 12 to 15 members in this work group. And diversity in the work group is critical. And I mean diversity in all its sense. In other words, people from different backgrounds, people from different specialties, people from different levels in healthcare provision. Next will be conflicts of interest. So conflicts of interest are obvious. People know what conflicts of interest are, but the bottom line is, do you have an interest that will secondarily impact the primary interest, which is the production of the guideline? Whether that is financial, whether that is something else. Do you have a conflict of interest that will impact that? This has to be documented for all members. This will ensure transparency and validity of the document. And importantly, not just individual, but household conflicts of interest become important. So of the entire organization, the people with the strictest COI requirements are going to be people who serve on a guideline. They will be required to have the least amount of conflicts. There are a number of different types of conflicts. These are three main buckets of conflict. The first one is relatively straightforward. Is there a financial conflict? Do you own stock in a company that is relevant to the guideline being addressed? That's going to be an issue. Are you an employee of a company? Or are you a board member of this company? Do you have financial stake here? Those are obvious. People understand those. And those are generally fairly rigorously reported. Non-financial conflicts of interest are also important. And people generally understand these. And these are unpaid board memberships on a company or an organization. There may be religious or other political affiliations that may impact guidelines as well. You can think of pregnancy-related guidelines that may have these types of implications. Those conflicts ought to be declared as well. The hardest conflicts are academic conflicts. And there is really a rigorous debate as to whether these are relevant or not. What I mean by academic conflicts are that if you are asked to write on a guideline on status epilepticus, and you've spent your entire life researching status epilepticus, is that a conflict of interest? Or are you a content expert? So you can take this debate either way. But you can also imagine how this becomes a conflict of interest for you, because if all of a sudden your life's work is not included in a guideline, that may impact you very negatively. So these are all things that need to be considered. Certainly, academic conflicts of interest are the hardest to deal with. And those are the ones that we often do not address very rigorously. So just because you have a conflict of interest does not mean that you cannot participate in a guideline. These conflicts of interest can generally be managed. Generally, you're not conflicted for the entire guideline. But if there's a specific part of the guideline, you can recuse yourself from that. The people who cannot have conflicts of interest, any conflicts of interest, are the co-leads of a guideline. They cannot recuse themselves from any part of the guideline since they're leading it. So they are not allowed to have any conflicts of interest. Different societies will have different guidelines as to how conflicted their guideline panel can be. In other words, some societies may say half the panel should have absolutely no conflict. Others will say two-thirds may, or rather one-third cannot have any conflicts. That is determined on a society-by-society basis. And again, academic conflicts, we generally are not managing. It's very hard to try to figure out what to do with academic conflicts. Next up is protocol development. This is sort of how we are going to do this. The elements of a protocol are listed on your right, and it talks about all the different things that are going to be required. Perhaps one of the most important things here is the PICOs, and I'll talk about that in the next slide. But included in the protocol is going to be who's on your committee, what conflicts they have, how you're going to search the literature, and finally, how you are even going to disseminate the information that you come up with with your guideline. How will you try to get this implemented in clinical practice? There are development tasks of the protocol that are assigned to committee members. Typically a methodologist or a librarian is involved at this stage, and importantly, there is a financial commitment by the society or by the group that's sponsoring these guidelines. And that is not an insubstantial commitment. That is in the order of many tens of thousands of dollars per guideline. The basis of every guideline is typically a set of PICO questions. And PICO stands for Population, Intervention, Comparator, and Outcome. You can also have TNS, PCOTS. This is timing and setting. Not every PICO will have the timing and setting, but the first four elements are going to be there for every guideline and every question asked in a guideline. PICO questions can be therapeutic interventions, they can be diagnostic evaluations, or they can be prognostic evaluations of a disease state. Any of these can be crafted into a PICO that asks about a population, what intervention will happen, what it is compared to, either nothing or something else that's a gold standard, and finally, what is the outcome that is of interest. The protocol then is approved. The approval process is generally several months long. You will first of all approve it within the committee itself, within the organization. There will be typically a public comment period as well where interested parties can comment on your protocol. And once you get these comments, you will then revise your protocol to create the final protocol. Once all this is approved, then the work on the actual guideline begins. That work will start with identification of the evidence. Typically, you will need to search not one, but several databases. You'll typically need a librarian to do this. There may be a methodologist involved as well. This process is typically described in the PRISMA methodology, and I'll show you a slide on that in just a second. And that talks about how you should analyze, review the literature, how you can discard articles that are not relevant, how you finally get to the ones that you are going to use. This is the PRISMA paradigm. It stands for Preferred Reporting Items for Systematic Reviews. All systematic reviews should follow these PRISMA guidelines. I'm not going to read over these. You see a QR code here. This will take you to the PRISMA page, and you can certainly review these. The purpose of PRISMA is to have a very standard methodology for systematic reviews, and the purpose of having a standard methodology is reproducibility. You want your guidelines to be reproducible by anyone else who follows the methodology that you outline. And that's sort of why PRISMA is important, so that you can outline that methodology. There's a standard paradigm that someone else can replicate. Once that is done, typically your systematic review will be registered in what is known as PROSPERO. PROSPERO is the International Registry of Systematic Reviews. This is critically important because this establishes your systematic review as legitimate. It is searchable for anyone else to look at. When others are doing systematic reviews, they can find your systematic review so they don't duplicate the work you're doing, and before you start your systematic review, it behooves you to look at PRISMA to see if your systematic review has already been done. That may save you a lot of time. The PROSPERO website will retain certain key features of each protocol that are searchable, that must be reported by anyone doing systematic reviews that wants to report to PROSPERO. This is maintained by the National Institute of Health Research. Once you have done all of this, you start to classify your evidence. You have determined which papers you're going to use, but now you must figure out what is the certainty of evidence that these papers offer. You obviously have randomized, hopefully have randomized controlled trials, but that may not be the only thing you have. You have observational studies. Randomized trials will hold a certain value. Observational studies will hold a lesser value, but not every randomized trial is excellent and well done, and of highest quality, you will need to have a risk of bias assessment. You'll determine the quality of the evidence and either downgrade it or upgrade it depending on what you find. The same thing applies for observational studies. Some may be very good and may be upgraded. Others may not be as good and may be downgraded, and what you see to the right of your screen are different reasons why something may be upgraded or downgraded. This paradigm should be done in a relatively, not relative, this should be done in an organized manner, and there are different tools available for this. The one that's being used often, the one we're using in AES is called the GRADE tool, and that is grading of recommendations, assessment, development, and evaluation. You see that on the last bullet point. After that is done, you will formulate your recommendations. These must be clear, concise, and evidence-based statements. They should be directions for delivery of care. Remember that people will just read the final statement. They should be declarative. They should use words like should and recommended rather than may and consider if you've got the evidence for that. So it's ideal to have definitive statements in your recommendations. At this stage, you must also consider patient preferences. And again, Renaud alluded to this. One of the Institute of Medicine's recommendations for any clinical practice guideline is to incorporate what patient preferences are. An example of this might be withdrawal of antiseizure medications after epilepsy surgery. There clearly may potentially be evidence supporting one or another course of action, but patient preferences are going to be key in making that determination as well. Finally, you're going to be writing this, and there's a number of steps. I won't read through these, but bottom line is that the writing committee creates the initial draft. That draft is approved by the entire committee, by the Guidelines Committee, presumably by the board of directors of your organization. There's a public comment period. There's a revision of the guideline based on the public comment. There's further approvals and then submission to the Journal of Interest. Typically, as these guidelines are being proposed and worked on, you've already identified the Journal of Interest that you will be submitting to. You've already had a discussion with the editor-in-chief in terms of what types of items are going to be needed and the writing is done with that in mind. Typically, most journals will readily accept clinical practice guidelines that come from societies like ours. These are high-impact, high-value publications. If there is a joint guideline that we are writing with another society, there typically will be simultaneous publications in that society's journal as well. I won't say the last, but one of the final steps is implementation. The implementation is the fun part of the guideline, but a very critical part of a neglected part. It's relatively straightforward to create the guideline. It is much more complicated to get it implemented into clinical practice. And there's a number of guidelines, there's been a number of studies that have talked about how guidelines have been produced, but have made no impact on patient care. And that is really where guidelines fall short. And so one of the things that you must do in a clinical practice guideline development is figure out what is your plan of implementation. How will you get the clinical practitioners to use your guideline? And I don't mean epileptologists who may understand very well why this clinical guideline is important. I mean general neurologists and primary care providers who may not appreciate, may not have the time, may not have the finances to implement these guidelines. With the tools we have in electronic medical records now, implementing guidelines is becoming easier because they're built into the electronic medical record and that facilitates the use of clinical practice guidelines. Updates are critically important. This is towards the end of your guideline. You've finished your guideline, but there must be an expiration date on your guideline. No clinical practice guideline is going to be valid forever. Data about what you've written about will change. And you must determine, depending on the topic that you have written about, how quickly do you expect that to happen? For example, we're working on a status epilepticus guideline. We'll create a guideline this year or next year, but we can be assured that in three or four, maybe even two years, there will be other clinical trials that will report other drugs or other treatment modalities that will require an update to our clinical practice guideline. So that's sort of the things that we will want to think about as we determine the expiration date. And then once you determine that, you figure out how often you will be re-updating your clinical practice guideline. None of this can be done without appropriate financial support. And I mean not just a few hundred dollars. These clinical practice guidelines require tens of thousands of dollars. Oftentimes, the cost of a guideline is determined by the number of PICO questions. And you can imagine that the longer the guideline, the more comprehensive the guideline, the more expensive it becomes. And that's why societies in general cannot have an unlimited number of guidelines. And this is one of the major issues surrounding guideline development. So in summary, we have gone through this journey. We have produced a guideline starting with a guideline proposal and ending with implementation. I'll wrap up by reminding you that clinical practice guidelines provide evidence-based recommendations that improve patient care and reduce variability in care delivery. And that is key, is reducing the variability in care delivery. These are informed by systematic review of evidence and the assessment of the benefits and cost of the intervention that we are proposing. There is a well-defined process in assessing the data and formulating recommendations. Implementations of clinical practice guidelines are key and critical, and updates are necessary. And don't forget, there needs to be a financial commitment. And so moving forward, once we start doing these more regularly, if guidelines are not being accepted by someone who has proposed them, it may well be because of the financial commitment already in clinical practice guidelines.

clinical practice guidelines

evidence-based standards

systematic reviews

PICO questions

patient care improvement

methodology adherence