Welcome to the performance management for signal systems conference call. All lines have been placed on mute to prevent background noise. After the speakers remarks there will be a question and answer session. If you would like to ask a question press star one on your telephone. If you would like to withdraw your question press the pound key. You may begin. Inc. you. Hello and welcome to today's national transportation operations coalition talking operations webinar on performance management for signal systems. My name is Jocelyn Bauer and I will give a brief introduction to the Web conferencing environment before turning this session over. This webinar is the fifth in a series of development courses offered by the national transportation operations coalition on signal systems. To access the recordings of previous signal courses you can visit the webinar archives page of the national transportation operations coalition. I would type that into the chat box. Our webinar is being recorded. It will last approximately 90 minutes. The format will be different than previous talking operations webinars. Instead of having several presenters we will have just one instructor and it will be more interactive. During the webinar if you think of a question you can type it into the chat box on the left-side of your screen. Make sure you send your question to everyone rather than just a% or is. The instructor will address the questions as he goes. A file containing the webinar will be posted within the next week. Attendees will be notified of the availability of the presentation of this webinar by e-mail. We encourage you to direct others who were not able to attend to access that recording. The presentation is available for download in the download box on the left-hand side of the screen. Click on the name of the file and click the button at the bottom of the download box. At this time I would introduce our instructor for today. Gary Thomas is a research engineer with the Texas transportation Institute. He has taught barely 70 workshops and seminars on a wide range of topics including IT a standards, topics about design and operations, intersection safety improvements and freeway operations. He has been a national highway Institute instructor for nine years. David, -- very start when you are ready. Thank you very much, Jocelyn and I appreciate the opportunity to be here today. I'm going to say hello to everyone out there and hopefully you can put a face to the name. I won't have it appear the whole time because it's more distracting, but I wanted to say good morning or good afternoon and possibly good evening if we have anyone dial in from other parts of the world which is not unusual. I'm glad you were with us today and more people are logging in as I speak. We will get started right now. Please go ahead and put in questions as we go along. I try to keep my webinars and a real classroom flavor and address questions as we go along rather than to think back. If I see things come through I would try to answer them as we go along. Jocelyn will answer any questions of a technical nature like regarding where I download things are weird to see this later. Any subject matter I will answer on the fly as we go along. I think a lot of you are listening on computer speakers. If you happened to dial-in, we will stop once or twice during the webinar to give you a chance to ask over the phone if you prefer. This is performance management system for traffic systems. They target audience for this webinar as directors of traffic engineering, and traffic operator's, practitioners responsible for day-to-day operations for signal systems and decision makers responsible for budgeting decisions regarding signal systems. There is a pretty wide swath on our target audience. Hopefully you fall into one of those categories and you will get something out of today's webinar. I am with TTI in College Station, Texas. I frequently teach for not only IT and FHWA but the national highway Institute as well. I just got back from Denver. Also IT and other organizations and active in TRB as well. Let's talk a little bit about what I hope you get out of this course today. I have got for learning outcomes. One is you should identify the elements of a performance management system and describe the benefits that can be gained from using when. Explain how performance management system can be applied to traffic signal operations. Identify measures and supporting data sources for traffic operations and explained ways to integrate performance measures into your agency process. As we go a long, you will see in the bottom that I have identified which learning outcomes each slide applies to so that can help you keep track of where we are at. You can download the slides in the file share box. I'm going to take a few minutes to look at performance management in a generic sense and not necessarily specific to traffic signal operations. This may be a little bit of a review for those of you who are familiar with what a performance management system is and can you use one. I imagine there's a few people out there who are not familiar with performance management systems and I want to bring everybody up to speed. Performance management concludes activities that ensure that goals are consistently being met effectively and efficiently. Performance management can focus on the performance of an organization as a whole. You can focus on the department, an employee or even the process to build a product or service as well as many other areas. I'm going to take a few moments related to performance management and personal areas so we can see a direct correlation between performance management in the workplace as well. What is performance measurement? I'm using a slightly different word, what is performance measurement as opposed to management, we want to look at performance measures. I have got three definitions. One is the dictionary definition, I don't know if I took that from Wikipedia, I don't know. A process whereby an organization establishes the parameters of programs and acquisitions and reaching desired results. There is the definition of performance measurement. If we look at more specific to transportation, a simple way to define it might be to say a way to quantify how well a transportation system is working. FHWA has an added definition to that and they said the use of statistical evidence to determine progress toward specific defined organizational objectives. The operative word is the use of statistical evidence. Those are pleased of looking at it. Performance measures can be hard and fast measured parameters such as pavement surface smoothness, if you're a hard pavement person. Travel times if you're in traffic operations, that can be something that can be measured. There could be measures of customer satisfaction which taking those two, the pavement surface smoothness, something that is more customer satisfaction would be the perceived ride quality of the pavement. It's not something that can be hard and fast measured like a device but it's going to be more of a soft measurement or where travel times can be measured, on-time arrivals, perceived delays by a driver. They may not be measuring it with anything but they might say most of the time I feel like I am late for most of the time it seems like my travel times vary or it's pretty consistent so that would be more of a soft measurement. Regardless of how they are measured, performance measures can be used to provide feedback on how where the system is performing, both from the user perspective and an operator perspective. I see something that the checkbox is covering up the slides. I'm not sure why that would be the case. It doesn't appear to be on mine. If you're on a computer with a small monitor, sometimes that causes problems. You might want to try maximizing your browser window. Major contributors to the inconsistency found in traffic signal operations and maintenance budgets can include some of these things. A lack of clear guidelines that describe traffic signal operations and maintenance activities and resources required to spark these activities. Also there may be a lack of documented objectives and performance standards. If we don't have them, how do we know if we have met our goals. We need them to be documented. Another roadblock is funding mechanisms are geared more toward project development that operations and maintenance. I think we have kind of seeing a little bit of a shift in the last five or six beers that we are seeing more funding geared toward O&M that we didn't see 10 years ago. In smaller organizations you may still struggle with that about meeting that ongoing funding for measuring how well your system is performing. Part of the reason for that is it's not as exciting to cut the ribbon on a performance management system as it is to cut the ribbon on a new roadway or bridge or something of that nature. The governance structure and division of responsibilities among numerous agencies at the federal, state and local levels can cause roadblocks along the way. The proprietary nature of sick assistance industry, both hardware and software can cause issues. I think we are seeing a little bit of improvement there especially as we start adopting ITS standards and things like that. Open source, open platform systems using the standards, things like that. Toward viability, I think we are familiar with that. Some are worried if we show our system is performing poorly that we are opening ourselves up to some liability. I don't have any particular instances to give you fair. We also have equipment issues. Some of us are dealing with old equipment that simply cannot provide this type of data we need. Different controllers, detection systems. If they have been after a wild, detection systems may not be up to par with what we need a maybe we are just now moving into other systems that are more reliable. Wouldn't deliverables and the project funding cover O&M elements and procedures? Yes, if that isn't there. If that is part of the project scope as a whole, and we are seeing that. A little bit more of a shift in attitudes. 10 years ago you might have been hard-pressed to see a lot of the O&M stuff in the project scope of work but now I think we are seeing that a lot more so that is a good right we are taking. Thanks for the question. Let's turn to more about why you should use a performance management system and I have posted a lot of different reasons you might. All or some may apply to your particular situation. Legislative mandates is one. Many years ago for those of you have been around a while, back in the days of 1991 legislation that specified the number of different management systems that were required and I would be hard-pressed to come up with five or six right now but pavement management systems is one of them in addition to others. They even actually got taken out but I think it got the ball rolling for our need as system operators to look at managing our systems and performance management. You may have some state mandates that require new projects to be monitored, that might be one reason why you would have to move into a performance management system. The planning processes including budget and funding allocations, you may be required, maybe your local jurisdiction, if you're working for a city, might have an ordinance in place that says in the planning process you must include performance management. Maybe your organization has quality initiatives they are looking at in performance management systems and that would be the way to monitor those. Congestion management systems and evaluations, ITS operations and evaluations, safety management systems. One that I came up with later I don't have on the side is dealing with permit processes for commercial development. We may be looking at how a system would perform as you get more development on an arterial street, for example. That might be one additional reason. There may be others I haven't included. At any time if you think I left off something, I would encourage you to put something in for the good of the class. I like to treat it like we are in the same room. I want to hear from you if you have ideas to add. There will be a couple of times I'm going to ask you some questions. I would try to give you a heads up before you do that. Transportation asset management would be and it example. If you are dealing with certain assets, I don't know if you're thinking about transit and how you would manage your assets and buses and things like that. There was a mention about pavement management systems. Street furniture, bus stops for traffic signal poles. Whatever gets measured gets improved, that's a good point to make. We need to measure something, you can show to the decision-makers, maybe it's a city council or city manager or a straight transportation commission, whoever makes the decision about budgets, when you show we are measuring this, they will say okay, we would steer more funding toward that area and hopefully measure it by next year it will be better. Thank you. Somebody else mentioned about learning from your actions and expanding knowledge base. If you implement a new traffic signal timing plan, if it's not measured, you don't know if you have done any good and you can learn from that. There's a lot of benefit as well just because you are forced to use a performance management system to through mandates doesn't mean you're not going to get numerous benefits. Good theaters keep their organizations focused on the highest business priorities. Agency leaders setting that agenda helps motivate staff. If you are in charge of some type of system and you don't give specific direction to your staff, they will lack that motivation. How do they know they're doing a good job or operating system that best it can be. Objective performance data helps them understand challenges. I often find strong performance you verges with day-to-day business processes are aligned with well thought out agencywide strategic priorities. When things come together, they line up with what you are doing with the strategic agenda. The evidence is there that we see a stronger performance. Accountability is certainly a fact of life, certainly for public agencies. Transportation agencies that ignored the expectations of there elected officials and citizens just in general, they run the risk of adversarial relationships that drive up the risk of negative policy mandates and reductions in funding. If we are not being held accountable, we have lots of people looking over our shoulders these days. People are very concerned about use of public funds and tax dollars and where they're going and not being squandered away. This helps having a system in place and it will improve your accountability to those people. It can provide best better customer service as well. You can develop tools to predict future performance here and in the airy of pavement management, a lot of work has been then in that area where we will able to start predicting our performance of payments and understanding the opportune time to come in and replace pavement or we ability pavement. The same can be done for assistance. If we have an idea of how often we need to look at retiming signals. We read time signals and start measuring things like travel time and delay, we will probably see delays increase over time. We will see travel times increase over time, especially in areas that are growing. By looking at that and measuring things, either on a daily or monthly basis or an annual basis, we can look at when we need to do this again. You need to think about that. Hopefully that can help transform public images and looks like I have an incomplete sentence there, I think it is supposed to say risk to using public funds and kind of getting back to that accountability. Also data archiving is not mentioned that they are. If you properly archived data, transportation agencies can use that historical data. They can make better use of those limited resources. Let's talk a little bit about the structure of performance management systems. The thing to remember about a performance management system is that it's only a part of a larger process that includes a strategic planning component which are the green boxes and circles in this particular diagram. Were not going to talk about them today. We are more in the middle and talking to the purple area, the performance management system with things around it and below. The areas of the Orange are not going to be addressed today in this particular webinar. In doing some research and surveys, we found there are few traffic signal programs that have a documented strategic plan. There are some out there that it is not widely used. Performance management systems is a continuous cycle. It consists of four components, selecting measures, sitting targets, making decisions and evaluating the system. The third of these steps, using performance measures to make decisions is the primary focus of this webinar but we will talk about the other things as well. The two biggest forms of performance based decision-making that influenced by a system are those that address resource allocation, which could be rephrased doing the right thing and resource efficiency which we could call doing things right. This is where the measures get back to the audit process that may exist doing the evaluation of the system. Performance management is the application of data analysis and innovation to support these decisions we are making. This is taken from a report about performance management structures if you Google NCHRP, usually report 660 will come up. Let's look at the first one, let me go back one slide, were going to look at selecting measures and were going to start there and go a round the performance management system. The strategic plan should serve as the guiding document in selecting performance measures. Measures should reflect agency goals and objectives, provide data needed to answer the question of how we are doing. The data should answer that question. Your measures should be outcome oriented, much like the learning outcomes but this webinar. You should be able to explain things and to find things when you are done today. Their measures should be outcome oriented, meaning they examined the impact of decisions made rather than the amount of resources being devoted to a particular practice. One question going to ask, if you are with the public sector agency, does your organization have a strategic plan for its traffic signal system? Jocelyn, can you do a poll off to the side? Public-sector agencies, does your organization have a strategic plan for your signal system. The measures an agency chooses to collect and track data should be specific enough to directly address the objectives set out in the strategic plan. You will want to identify measures that are relevant and that is sometimes a tough one to come up with. What are relevant objectives and what may be relevant to the traffic signal engineer may not be relevant to the mayor or city Council. We went to identify those measures that are relevant to the achievement of agency objectives. Data for the measures should be reasonably easy to collect and work within the agency operations and maintenance. It's important to employee measures that can be tracked incrementally and compared against performance targets. It should be easy to collect and easy to collect fairly often. Performance targets are often very ambitious and measurable incremental parsers may be an indication the agency is doing the right things but they need to step up efforts. I'm thinking that more toward life in general. If you've got certain performance measures you want to meet, you can have a very ambitious goal, perhaps it's the brand-new year, people say they're going to lose weight, again, you might set ambitious goals that you need to do something that you can track incrementally and know you're making progress instead of waiting to the end. We will talk a little bit more about doing the targets in just a minute. As data collection and analysis improves, new measures will become possible that better reflect a particular outcome than whatever measure is currently in use. Agencies should be careful to update measures only when the new measures clearly improve the decision-making process. You don't want to jump around from measure to measure. You want to be consistent and update them when you have a clear reason for doing so. Maintaining existing measures provides continuity and gives historical information you need to understand trends. A bunch of you have been answering the question. Most of you said no. 11% of you were not certain and those are for you in a public agency, and we probably have a lot that are not public agency members in the room today. Thank you very much, Jocelyn. A lot of our agencies out there that are participating don't know if they have a strategic plan for its traffic signal system. It's something you might want to think about doing. If that's not your particular job, you may want to talk about those that are responsible for doing something like that. One of the first steps in designing a performance management system is to bring together stakeholders to discuss and select performance measures. The measures should be meaningful to the typical driver. For example, agencies oftentimes focus on traffic's performance measures such as delays and stops and we base this on aggregate measures, overall delay, average number of stops per day, something like that which is not perceivable by an individual motorist. Things like travel time and reliability hold more meaning to them because they know what their travel time is where they know if it fluctuates wildly, that on some days it takes an extra 50% of their time to get to their destination, that they perceive quite readily. The stakeholders should refute a new strategic plan to documents that are relevant to the operation of the signal system. You would want to identify data elements collected by the agency. You may or may not use them depending on how accurate and reliable the data is that you want to start somewhere. There may be data elements not currently collected that you have the capability of collecting. As air controllers get more advanced and detection gets more advanced, we can start collecting other things. We are seeing an increase in Bluetooth technology and using Bluetooth to collect data. I can talk more about that with you separately if you want, but a number of jurisdictions are starting to use Bluetooth capabilities to collect that information. Once you have looked at that data, what are and are not collecting, you want an assessment of equipment needs to complete anything, maybe you have data you want that you don't have the ability to collect and you want to identify that. Let's Move On to setting targets. There's a number of acronyms and ways to set a target. This is one I use a lot of times when describing about setting targets, they all very just a little bit. The acronym is SMART that the K7 stands for specific. Targets need to be specific in nature. For traffic operations, it might be improve travel times in the peak hour by 10%. It is very specific. They need to be measurable, something on a regular interval. As I pointed out, you may already collect some of this data but do not presently doing so or maybe you need new equipment to capture that particular data, but I target should be measurable. Ambitious, I think it's important to set ambitious goals, again whether it is a traffic signal system or your personal life goals. If you're looking at getting out of debt, sector gold to be ambitious, not outrageous, something -- be ambitious in setting or targets that be realistic in finding some middle ground. They also need to be time dependent. Say that you're going to do something, set a time to do that otherwise it's not going to get done, the same with performance management systems. You want to set time dependency on all of your targets otherwise you can say that is our target but I don't know how long it will take to get there and we will get there when we get there in which case, you will probably never get there. Let them be short-term and if so progress can be measured. Targets that support strategic planning objectives should probably look no more than none two or three years ahead. It is tough to set a target for something 10 years out because something will change a lot in technology. Even two to three years, technology changes pretty fast that we can probably do that and have a good target. We will come back to that in a few minutes. One of the first steps in designing a performance management system, we brought together stakeholders and now, how do we use those measures in decision-making? We've got strategic decisions that have been made and now we look at how we are going to allocate resources. How do we allocate resources based on those targets in the performance management system. You also have some programming decisions and operational decisions, these are all kinds of decisions you will be making based on the performance of your system. Operational decisions such as making changes to peak hour timing. As we get the information back, that may trigger decisions about peak ours spreading out, it used to be mostly from 5:00 to 630, but with new development, the towers are spreading out. That is an operational decision. Or when you simply change cycles or something like that. Human resource decisions can come from those measures such as how you are staffed up and how many technicians you have per traffic signal or 50 traffic signals, something like that. If you are growing area building a lot of traffic signals each year, you may use that to trigger human resource decisions, like hiring more staff or re- tasking certain people to do other things. Programming decisions, I think of more of when do we add signals, how often and when? Are we going to start doing things differently as the system expands and is travel times and delays go up? Do we start doing things differently, that is a program decision. A lot of these decisions, when you accuse measurable things, they came trip dirt in decision-making things you need to do at specific times. The performance management system itself should be regularly updated, and evaluated. You don't want to do it every month, but you probably want to add a minimum come back on an annual basis and look at how your system is working. The interaction between setting strategic agency priorities and selecting performance measures and targets will be to changes in both priorities as you data demonstrates more efficient courses of action. Another thing we see is simply the technological advances and feedback from employees and external sources, they may lead to a development as an approved measure or more per lap a collection measurements. Going back to thinking about this Bluetooth data collection that some jurisdictions are doing, five years ago we were not even talking about it. Three years ago there was maybe initial discussions about it being used and now, agencies are doing it. The stakeholders need to be educated that performance measures may vary period to period. Maintenance of the system is also valuable in evaluating the system. I haven't seen any questions come in so I think what we want to hear, maybe I would just take a minute and turned to the operator to see if anybody wants to answer a question over the phone. Operator, tell them what they need to do. If you would like to ask a question over the phone, press star one on the telephone keypad. We will pause for just a moment. Okay, star one to ask a question. So we don't have dead air, I'm going to keep talking. What we have been looking at up until now is talking specifically about traffic signals in some instances but that was kind of the overview of what a performance management system might look like and in a moment going to look at specific traffic management plans. There are no questions at this time. Thank you, we will continue on. The most common system measurements, we will talk about those coming up. We would see if your question gets answered as we go along. Let's talk more specifically about traffic signal systems. It has been estimated in previous research had a proffered traffic that but timing can account for 5% to 10% of traffic delay. 295 million vehicle hours of delay on major roadways alone. Another study, congestion conferences the average peak traveler an extra 38 hours of travel time. That is from the 2007 urban mobility report. We just came out with the latest urban mobility report. I would check those numbers and maybe get those updated but it has probably gone up. We need to realize traffic signal system performance is critical to reducing delays and reducing congestion. This was taken from the outcome oriented performance measures for the management of signalized arterial capacity. That is completely dedicated to performance management systems and traffic signal operations. If you don't have a copy at that and it's something you want to get into, I would encourage you to seek out that document. While it is research oriented, I find some of this stuff is very practical. You can put it to use in real life situations. This diagram illustrates what would be a typical organization approach to signal timing and maintenance. Based on objectives, data are collected for specific times of the day. There are various software packages that will design cycle split an offsets geared toward those particular objectives. Again, we might set priorities, look at getting data and then we might model it using some particular software. Then you would do the design and documentation. You can put the data, get the data out, deploy it. He might do a little bit of feedback here. You look at the timing and documentation and any good traffic engineers would simply not take those timings and put them in verbatim out in the field. A good traffic engineer would look at them and see if it makes sense and probably go back to the software and make some adjustments, perhaps something is not real estate and won't work, come back in, get some designed. Once you're satisfied, move to the deployment stage and hopefully, the valuation. Make sure what to put is what you expected and that things are improving. Then hopefully we have the feedback cycle. Sometimes we take too long to go back and the evaluate and come back at look at doing a different timing design and documentation. We have a significant amount of quantitative feedback in this area and the timing design, but very little in the deployment area. Inherent weakness these can lead to operational deficiencies such as the data collected. We are typically very concerned about peak hour travel. We forget to go out on Saturday afternoon. Timing plans, they are oftentimes static. Most agencies do with static timing plans meaning they don't change on-the-fly. They may change to a new traffic plan at a particular time of day but they are not responsive to traffic. The data is collected sometimes on the order of years or customer complaints. We might take them two years to evaluate something again because it is so labor-intensive to make travel time runs. Different ways of collecting data are making it a little easier, but back in the day there was a lot of hassle to go out and collect this data. Even your data can you the same deficient results from software programs. It's not always going to solve our problems or make things better, but you never no. We need to look at more of this evaluation. The performance management system where this is collected automatically can help out quite a bit. The question becomes, is there a better way? Again, data and information we have developed over the years. The national report card, the last one, this is 2007. The new one closed a month or two ago, hopefully some of you participated in that in doing yourself reporting. Self reporting in the last one said traffic monitoring and data collection got a "F." The report said it is the greatest potential for improvement. 43% of those that reported little to know regular ongoing program for collecting and analyzing traffic data. Half of the agencies do not assess the quality of the data collected. This is self-reported information so it's agencies regularly at getting their shortcomings when it comes to operating and maintaining traffic signal systems. Before we explore the answer of can we do better, let's look at the principles of performance management as it relates to traffic signal systems here it the following principles provide the basis for an objective -based traffic signal management program. First of all, we want clarity up of jackets. The middle circle that is out of focus, want to have clear objectives. The attainable performance evaluation should be linked to those objectives. Standards of performance, for example, scoring well on the signal timing report card, that is a signal of performance. Based on objectives rather than agency activity, especially for arbitrary frequency. Research requirements based on objectives rather than industry norms. All based on clear and consistent communication with policymakers and elected officials, and also based on systems engineering process. If you don't know much about systems engineering, I encourage you to find more information on that. They really go together. The two our direct linked -- the two are directly linked. I will try to look at getting up a new PDF when we have done today. Drivers have simple overall expectations, yes, they are complex beings and hard to predict and control, but in general, customers have one primary expectation. They want to drive to their destination at their desired speed, minimum attention or at least they want to be treated fairly and particularly so they can plan their day with a minimum of uncertainty. That is a customer expectation and how you might drive your objectives might come from that. That is not going to vary from location to location. That is your customers and what they are thinking about. The agency of objective is slightly different. They take a little different approach. It can be very simple. This is an example. We would do our best to avoid making drivers stop and win we must make them stop we will delay them as little as possible within the context of safe operation. That is an agency of objective which should correlate well with customer expectations or customer object it's. From there, we can develop a good strategic plan. What are some of your objectives? You can just type them in. If you want to chime in on any of your objectives. What are some of your objectives as an agency or public agency for your signal systems? 100% operation. No downtime, I guess. Your system operates all the time and are not having any malfunctions. The objective. A huge issue for signal systems is addressing performance for pedestrians as well. Absolutely. I've been more vehicle centric so far that absolutely, pedestrians, what are their objectives for their expectations? They expect a safe environment that is reasonably direct, that is obvious and gives them time to cross streets and things like that. Good point. Minimize total delay and intersections. That is an agency of objective. It doesn't mean much to the driver. All I know if I am being delayed. You can have those other objectives, I'm not saying you can't have been. Able to respond to complaints, you might have an objective to respond to all complaints within a certain timeframe, at least begin to address or analyze them. Monitor the health of your detection and preemption priority systems, that's a good one. Keep an eye on that. Knowing whether your detectors are failing. If you have a lot of loop detectors and you are in an area at the country that experiences a lot of freestyle cycles, those detectors are very susceptible to damage or in areas of extreme heat when you have a lot of asphalt, it will break loop detectors. Video systems are not perfect either. You want to make sure those are operating properly. Cameras can move over time. You want to make sure video detection is healthy as well. We do some number of complaints. System approach for the network took times, not just -- more aggregate time measures, not just the speed between these two streets. Minimize minimum green time when using detection so there are not any unnecessary delays here it we do think delays in complaints, minimize response statements I preventive maintenance. Able to update and optimize traffic timing quickly and efficiently. Snappy operations, I like that. Those are your objectives we do need to flush out the little more to be a little more specific on how you measure a snappy operation. I know you know that, but I like the way you phrased that. I mentioned this relationship between performance measures and systems engineering. Once you have agreed on the agency and objectives, it is time to develop performance measures. Each performance measures should be linked back to at least one objective and similarly, each objective should have at least one performance measure. It can have more than one, but the objective can be one too many and the performance measure should link back to a one to one agency of Jackie. If you're familiar with engineering rosters, this probably sounds familiar. We develop user needs first and then functional requirements. Every user need should have at least one folks are requirements and every functional requirements she traced back to one user lead. Functional requirements must be specific and functional requirements should be measurable and subsequently testable. The engineering process can't be applied to a traffic signal performance management process. If enough for me with system engineering, I would encourage you to look that up. It is a lot of good material out there with system engineering as it applies to ITS projects. Let's look at an example. Performance measure, you see have got my topics appear. Specific, measurable, ambitious, reasonable and time dependent. To reduce travel time, if we took that as our performance measure, it probably doesn't really meet all of them. I would argue it is not specific ended isn't measurable and no ambitiousness to it. There is no reasonable notice and time. We do so for normal traffic conditions, that might be a little more specific but it probably doesn't meet the other ones. It is specific. Reduce travel time for 25% for normal traffic conditions. We are getting closer. I would argue that is very specific, measurable and ambitious but it probably doesn't meet the criteria of something that is reasonable or reachable. Reducing travel time by 1% for normal traffic conditions. Now we have just switched it around. It's not terribly ambitious but certainly is achievable without too much trouble. Reduce travel time by 3% for normal traffic conditions when signals are be timed. Now we are getting something that one could argue does need all five criteria. It is specific, we can measure it. Maybe it is ambitious and totally it is reasonable. It gives a little time dependency when signals are retirement. One might argue in the next six months they would be even more time dependence. It could still probably be a little better. If we continue with this example, we ask ourselves what we need to measure in order to determine if our objective has been met. The obvious answer is travel time. The next question becomes, how are we going to measure travel time? We can manually collect the data, and this is probably the most common we'd use. It has always been the most intensive, labor-intensive, time intensive. Analyzing the data can be very intensive. If it's manually collected, it needs to be put into something, a spreadsheet or whatever. There are lots of downsides doing to manually collected data. You can automatically collect the data. It may allow us to do it easier and it is working. I keep mentioning the Bluetooth example for collecting travel time data being used around the country in several different cities. Simulated data using simulation software. I'm not going to go into any details of particular software. They can give estimates of travel time but the results are only as good as the data that is put in and the assumption of the model algorithm. You can certainly do simulated data, but it may not be the most reliable. Finally, observation. Direct observation is a good way to get a quick feel of the data that represents reality. Brian asked about travel time, that is the complexity of the real world. I was being again driver centric and the travel time from 1.2 the next and not being linked specific here it is we do collect it automatically, people will drop off the system and vehicles will drop off the system. You have to be very specific. Who's travel time are we talking about, and you are right. We can be even more specific. In the case of more freeway operations, you have got more specific locations where people can enter and exit the system. Texas happens to be experimenting with this on interstate 35 as they do reconstruction and looking at collecting travel time data using Bluetooth that will feed back into a system that says somebody minutes of delay between here and there and that is displayed on mobile devices and things like that. I am being very open about travel time, it is not very specific. We need to identify resources. Once we have identified the target and data we are going to collect, are we using manual collection? That is certainly the most time and labor intensive. I did a long time ago. I sat out on the side of the road and collected this data. It is very difficult to do. Employees driving routes, having been keep a log or something like that. It is not real-time but it can help you identify historical trends. I think you can see the downside of that too well. You can do timestamps automatically. You can have vehicles instrumented with GPS and Bluetooth readers. You need to identify resources you have or what you need. I'm going to skip over this slide because I think we already talked about it and that is the advantages and disadvantages to that. We will come back to it if we need to but I think we covered that already. Test runs, these are manual data sources you can look at. Employees at the agency can perform test runs and I have certainly done that. I had a GPS unit on my vehicle and made travel time runs to see what type of improvement I could make. The data is slightly skewed because I am not the average driver. I have a vested interest in the data improving and that might skew the way I drive. I might deliver I slow down if I know the signal is supposed to be turning green rather than go a couple of miles an hour over the speed limit. Test runs can be very helpful as long as you do it correctly and without bias. Citizen complaints was mentioned about an objective and that is measurable. We can measure the number of complaints we get per signal or the failure of complaints to keep pace with the growth. If you are growing and you are getting more signal systems but your complaint levels are not going up, that is a good sign. The prevention of a sudden and sustained increase in complaints would indicate the agency has not slipped in his measure. Various traffic studies in kilo patients. Make sure you collected without bias. Trouble calls, several of you mentioned that. Making sure to Texans are working or 100% operation. Whether they are triggered automatically by your system, we want to make sure we can track those and eventually that they are rare occurrences. Accidents, another good source of data. Certainly for helping long-term trends. Historically, the accident timeline is what I'm looking for, we sometimes don't get the information for a year past when the data was collected. When we see more and more automated systems coming online, we are seeing the time Mike reduced and that is a good thing. Accident trend is very important to monitor the health of your traffic signal control system. Hopefully you get that data fairly often. I already mentioned most of these for automated data sources. Loop detectors and video detection. Video detectors are being used more because the costs have come down. They are finally comparable to loop detectors. Traffic signal controllers are becoming more able to collect different types of data. Probe vehicles are still used occasionally and blue tooth technology. In the future, connected vehicles. Vehicles start talking to each other and the roadside, we're going to see more data coming in. I see sudden stops as the potential data collection. If you're getting the data back from connected vehicles that are saying 20% of the vehicles on this approach are decelerating very quickly, that might tell us we haven't offset problem. They are expecting it to turn green and it's not or something like that, I'm thinking outside the box of what type of data we want. In a broader sense of control data, independent area post affect performance so that effect can be related to cause, do you want to comment on that? I'm struggling a little bit, if you want to expand it would be happy to follow up on that. Examples of ratios, something that doesn't mean much to an individual driver, that can mean something from a systemwide view. There was some work in Arizona, they wrecked are fitted as signalized intersections with the goal of -- to its fullest extent. In addition to video from the cameras, they put it into phase call its to provide contact closure. The outputs were conveyed to an on court event -based data locker within the traffic controller. They were doing stuff inside the controller but this information, but they were able to get performance measures generated. That was compared with manually counted data. That is also from the same transportation research record 2192. Stuff implemented in the fields, I would encourage you to get your hands on this and it can be very practical advice. As your independent variable changes, that can affect your performance more in a cycle I guess is what you are saying. As one reacts to the other it kind of feedback and could possibly make things worse. As incidents go up, that is one measure but as incidents go up, it can also cause problems on your system possibly causing more incidents or something like that. They are all related, absolutely and you need to be careful about the whole causation correlation. Just because something is correlated doesn't mean there is causation. We need to be cognizant of that. Here is one on calculating queue lengths. Thus the concept of khaki relations out of the Denver regional Council of governments. This is a concept they used a detectors to calculate queue lengths. You've got advanced detector actuations. They measured the are prideful flow and profile of the current cycling's. You have got to stop our detection down here. The two profiles are combined to determine if there is an accumulation at the intersection. With the maximum queue length this has been estimated once for each cycle. They are using video detection here and here and they have got the advanced detection and the output detection and again going into the controller they can make measurements about queue lengths with the objective to minimize queue lengths during peak periods. Another example is collecting travel time. Using Bluetooth, for those of you that don't know much about Bluetooth, Bluetooth is a short-range communication that is very prevalent these days, mostly in mobile devices. A lot of cars these days have Bluetooth communications in the car back and talk to your mobile device. Technically you can do hands-free talking on your cell phone, however I will say there's no research that shows hands-free cell phone talking is any safer than talking on a device to hold up to your ear. By having this Bluetooth technology in there, which can be used simply for piping your music to speakers, that is another example. They give off an address, but sent out an address that is unique to that advice. These two can talk to each other. If we start back here, this can say that device at this address has just passed me by and it sends that information to the traffic management center. Later on this yellow car goes by this signal and a device in here sees that same address and sends it to the traffic management center and then they can't estimate the travel time for this particular intersection to this one. That is how we are seeing some of this being done. It is pretty much cutting-edge right now, some are doing it, but not a lot. Were going to talk about some of the factors for success. You want to begin by focusing on a clear and present challenge faced by your agency. Use performance measures to describe the problem and provide the most appropriate solution. You want to focus on your challenges. Bring managers and employees along and building their capability to use and manage the data. Make sure you have everybody involved. If you want to expand, start probably small. Expanded over time. Don't start with everything in the kitchen sink or you are probably destined to fail. Start small. Agency staff will take ownership of their work. Train your agency managers and employees to focus on the needs of a agency customers and balanced standard engineering and programmatic considerations so that the agency appears credible and capable. Sustained efforts over time by ensuring the program is not connected to a single individual or office within the agency. You want to keep it broad-based. Inshore product distribution of performance data to your legislators or whoever that may be, and the public as well. Build constituencies. A little bit about reporting, one of the overarching goals of performance management is to increase transparency and accountability. This is done through reporting. Collect the data you find and analyze all the data you want, but if you keep it to yourself, it was they are not being transparent. Frequent public reporting can produce numerous results. Building accountability and trust within your organization and its constituencies. It helps strengthen support for budget and program proposals. It promotes friendly competition. You may want to look at how you do that. Not putting employees against each other, you don't want that, you want to have them spurred them on. They are doing that, we can do that. Creating an expectation of continued reporting and incremental improvements which can solidify that program you it implemented. Here is an example of one reporting. This is more related to pavement, but this is a dashboard available to the public. It has got performance, safety, condition, finance, projects, management and citizens survey results. I need to redo to take a look at that. -- I encourage you to take a look at that. Start thinking about questions you want to ask and we will take them. Here is a good example you might want to go to on your own time. This was done that in 2008. It was very collaborative. These are the key questions they felt anybody should be asking. What types of data should be collected? How are you going to store this? What frequency? How will the information be disseminated? What information will be provided to the public? Some of their issues and challenges they came across, the capability to store recorded data for longer periods of time. If you are collecting video data, it is hugely intensive. Simple volume speed data is just numbers you are recording and it won't be much of an issue. You want to maintain good communication to your field devices. Educate the public. All of these things they identified as challenges they had to go through. Here is the concept of operations, part of the engineering systems process. They developed -- between the users -- and field devices appear. That was their generic concept of operations. That brings us to about five minutes till. I think we have covered all of these. You can look at them again. Feel free to put in some questions. For more information, transportation research record 2192. I'm trying to remember if it is available for free or if you have to buy it, you can go to the TRB website. A PDF may be available online. Any questions at this point as we finish up? If you are on the phone and you want to ask a question, star one would be the thing to hit. I'm not saying anything coming on the chat box. I'm going to look at the file and find out what a slide or two is missing. You can always come back into this room and download it and an hour or two. There are currently no questions on the phone line. All right, thanks again. Some people are typing so there may be questions coming in. Jocelyn, is there anything you wanted to cover before we end the call? Yes, I wanted to go over the -- organization, but this be a good time? Yeah, if you have a question, continue on and I will address it. Thanks, Gary. On the first side, you would see the member organizations of NTOC. We encourage you to go to the NTOC website that will be listed to find out more about these organizations. The NTOC website contains information about upcoming webcasts and contains a webcast webpage. We will have the recording and slides from today's webinar up within a week. For those of you who registered you will see that e-mail giving you notice that the availability of that recording and the slides. NTOC has two discussion forums, one focusing on high-level or strategic issues and the other focusing on ITS to climate and lessons learned. You can sign up on the website for the newsletter which is mailed out my e-mail twice monthly. That concludes all the information I have share. I don't know if there are any other questions you would like to answer. One is reminding me to correct the typos and I will do that. I think the DRCOG slide has the URL on it. I will double check before I send it out to everybody to make sure that link is still active. I've don't see any more questions -- I don't see any more questions. Thank you all for your time (end)