Continuing Education Program Evaluation
Chapter One:
Introduction
This report is intended to comply with the requirements of Senate Bill 1337, Section 13 which states:
The commission shall conduct a study on methods of measuring the quality and effectiveness of continuing education training programs under Section 415.034, and report the findings of the study to the 75th Legislature not later than January 31, 1997. The report shall include: recommendations for methods by which the commission will measure the quality and effectiveness of continuing education programs, a plan for the implementation of the recommended methods, and a means of funding the methods. In conducting the study and developing the recommendations, the commission shall consult with law enforcement organizations and advocacy groups with an interest in continuing education programs for peace officers.
The statute had as its immediate source the recommendations from the Comptroller's Texas Performance Review Report Advisory: Criminal Justice Education in Texas which was undertaken at the request of Senator Mike Moncrief and the Senate Interim Committee on Domestic Violence. One of the recommendations in that report was to study methods of measuring the quality and effectiveness of in-service training.
State law should be amended to require TCLEOSE to study methods of measuring the quality and effectiveness of in-service training.
This study should include recommendations for methods to be used, an implementation plan and a means of funding the recommendations. Furthermore, state law should require TCLEOSE to consult with law enforcement organizations and advocacy groups with an interest in in-service training areas. The study should be reported to the 75th Legislature in 1997.
In addition to the statutory requirement, there are more fundamental reasons to look at training impact. There is a tremendous amount of time, money, and energy being invested in these required continuing education programs.
If the legislature wants the Commission to develop and prescribe 20 hours of training for 56,000 plus peace officers and then evaluate that training for its effectiveness, it would make sense to invest in the course development and evaluation system. The legislature is providing $6,000,000 to the local agencies for delivery. If a peace officer's time is worth $10 an hour, the agencies are investing 20 hours X $10 X 56,000 for a total of $11,200,000. This does not include any additional investments from the local level for delivery costs that are above what the state has provided. Some at the local level have complained that the money received was insufficient. It still comes to a staggering $17.2 million which is 10 times the Commission's budget.
The fundamental questions that arise are:
The report will touch on these issues, but it should be remembered that these are endemic issues that have to be continually addressed if the continuing education programs are to be as effective and as efficient as possible.
The most fundamental requirement that should be met in developing a successful evaluation of a continuing education program is that the program itself be based upon a clear needs assessment. It is not logically possible to evaluate a program unless it is clear what the problem or need was that caused it to be created. In its simplest and most direct sense, if the problem or need that caused a program or intervention to be created is not improved then the intervention should not be deemed a success. (Borich & Jemelka, 1982)
Brinkerhoff (1989) has succinctly stated the importance of needs assessment to the evaluation process. "Where training is truly needed and reasonably well designed and delivered, it will automatically make a difference. Where training is not truly needed, it will not make any difference regardless of how well it is done. In either case, a solid needs assessment, not some narrowly construed, after the fact, experimental-design-based evaluation, is needed first." (pp. 12-13)
Chapter 415.034 of the Government Code was amended four times in the last session of the legislature in Senate Bills 80, 225, 1135, and 1337. The four different versions are contained in Appendix A. In summary, they require the teaching of various topics for different licensees regulated by the Commission. Those topics required for all peace officers are civil rights, cultural diversity, and racial sensitivity. Those topics required for peace officers, unless exempted by the law enforcement agency's administrator, are: child abuse and child neglect investigation, sexual assault, family violence, and characteristics of sex offenders. The statute gave the Commission the power to require reserves, jailers, and armed public security officers to take a course on the topics of civil rights, cultural diversity, and racial sensitivity. The Commission did this by Rule 217.9 contained in Appendix B. One other provision of the statute is that first time supervisors must receive supervisory training. The statute also required the Commission to prescribe the minimum curriculum for these areas, and to do so in 20 hours or less. An individual has two years to meet these requirements.
This past session of the legislature also explicitly gave the Commission authority to suspend a peace officer's license for failure to comply. A recent Attorney General's Letter Opinion No. 96-070 advised the Commission that it could not take action against a chief administrator's license for failure to provide the training as specified in 415.034. The legislature also has provided funds directly from the comptroller's office to local agencies for conducting this training in Senate Bill 1135. The two dollar fine surcharge for local agencies is at a rate twice the Commission's fine charge rate, and all of the local funds are appropriated rather than the 47 percent that the Commission receives from its fine surcharge.
Even though no additional funds were appropriated for this, the Commission developed and prescribed the curriculum for the required courses. Course 3939 Cultural Diversity covers civil rights, racial sensitivity, and cultural diversity topics. Course 3232 Special Investigative Issues covers the topics: child abuse, child neglect, family violence, sexual assault, and issues concerning sex offender characteristics. Course 3737 is designed for first time supervisors and covers the topics necessary for a new supervisor, including the above mentioned topics from Courses 3939 and 3232 from a supervisor's point of view.
As of January 2, 1997, the required continuing education courses had been taught to several thousand individuals. The numbers for each course are as follows:
This report will be divided into sections. Chapter One provides introductory and background information. Chapter Two will briefly review the issues of training evaluation and the literature on evaluation methodology. Chapter Three will describe the methodology used to gather data. Chapter Four will describe the results of a survey of the training providers which will provide a grounding in what is the state of evaluation among law enforcement training providers. Section Four will describe some of the alternatives available. Chapter Five will summarize alternatives, discuss recommendations, and propose an implementation plan.
The next section will deal with the general problems that are associated with training evaluation. Also included in the next section will be a brief review of some of the key ideas found in the literature of training evaluation. This review is necessary to place the current issues in an understandable context.
Chapter Two:
Training Evaluation Issues and a Review of the Literature
General Evaluation Issues
The classic framework for training evaluation follows the postulates of Kirkpatrick (Geber, 1995) in 1959 who defined four levels of evaluation to describe the alternative approaches. The Kirkpatrick Model approaches are: reaction, content, application, and impact. The four levels are:
Level 1. Trainee reactions to a course is the first level of evaluation. Commonly derided as "smile sheets."
Level 2. Level two training involves testing to see if they learned and what they learned. It usually involves a paper and pencil test.
Level 3. Measurement of level three is more difficult because it tries to ascertain if behavior was changed on the job.
Level 4. Level four tries to determine the impact upon the organization by looking at the bottom line outcome. Did the changed behavior affect the organization's business results?
A fifth level has been developed and used by some which is cost benefit analysis or utility analysis. Although, somewhat more difficult to do, it does translate into measurable units that drive most business decisions. Cascio (1987), Spencer (1986), Robinson & Robinson (1989), and Schmidt & Hunter (1986) have developed useful models. The focus is on comparing costs to the benefits obtained in terms of increased efficiency, effectiveness, or productivity.
Typically, evaluations have been done at levels one and two because of the time, energy and costs involved for more comprehensive evaluations (Gerber. 1995). "Penny-wise and pound foolish." People are so busy that they do not take the time to do deeper level evaluation, nor do they take the time to do comprehensive needs assessments which is a crucial prerequisite to effective evaluations.
Level two has another evaluation problem if it is the only system used. Besides the psychometric problems it reflects confusion about who is the legitimate customer of training. According to Brinkerhoff the true customers of training are upper-level management because training should serve the needs of the organization. In the courses required by these legislative mandates, the legislature could be viewed as the ultimate customer. They have identified the needs based on the input they receive from citizens. The citizens of Texas are the ultimate customers.
Needs Assessment Issues
One of the key concepts in the evaluation literature is the critical importance of having a total program design that begins with a needs assessment, followed by program development, then by delivery, and ended with evaluation. We will begin first with a review of an article by Brinkerhoff who has synthesized the work of many. (Brinkerhoff, 1989). He has identified this model effectively in his article, Using Evaluation to Transform Training.
One useful concept identified by Brinkerhoff is the tendency among trainers to be program-driven rather than needs-driven. "Professional trainers are likely to be most concerned with designing and delivering efficient training; such concerns for doing training 'right' can easily grow to supplant concerns for doing the right training." (p. 8) In this approach, training is administered as an activity rather than for business impact. There is a need to change to needs-driven training from program-driven training.
It is the results of training and the application of new skills, knowledge, and attitudes that organizations need, not training programs. Evaluation, by focusing on training results and related organizational factors (management support, for example) that impinge on results, can help the training culture become more results conscious. In essence, this requires that training evaluators pay more attention to needs assessment and to follow-up of training application and impediments. (p. 15)
Typically, not enough attention is paid to either needs assessment for training or evaluation of the training once delivered. Anyone involved in training cannot help but observe that the typical practitioner is so busy organizing and delivering training that "there is very little effort paid to needs assessment and other front-end analysis, or to follow-up assessment of training results. (p. 9)"
There is no one accepted process for conducting a formal needs assessment for every situation. Caffarella (1988) has identified ten steps in a formal needs assessment.
There are many techniques for data collection. Caffarella (1988) has identified eight of the main methods:
Some others frequently used are attending professional meetings, conversing with colleagues, and reviewing training materials or professional literature.
Not every problem or need is a training need. When a need has been identified, a determination must be made if it is a training need. The first determination to make is to answer the question as to whether the need is performance related. Is the need something that they are not doing that they should be doing? If the answer to that is negative, then something other than training is called for. If the answer is affirmative, then one must determine if the need arises because of employees lack of knowledge or skill. If so, then it is a training need. If not, it is probably a motivational problem best dealt with by some other technique.
Training cannot solve all of a profession's problems any more than education can solve all of society's problems. Caffarella (1988) has provided a matrix of alternatives to training to solve performance problems. (See Table One)
Alternatives to Training
TABLE ONE
Alternative for Job Specific Organizational
Individuals Alternatives Alternatives
Provide job aids (e.g., Redefine the job Improve selection
checklists, charts, methods and criteria
memory joggers, for hiring
reference aids)
Transfer or terminate Change the equipment Change the reward
the individual structure
Change the conditions Change the
under which the job is organizational
performed structure and /or
support patterns
Change the performance
standards
Communicate the
performance standards
differently
Eliminate the job
Evaluation Issues
Brinkerhoff (1989) has identified some inherent problems with training and training evaluation. These characteristics are problematic, and it is essential that evaluators be sensitive to them.
1. Skills, knowledge, and superficial attitudes and beliefs (SKA) are only a part of the larger human performance puzzle,..." It is impossible to attribute complete credit or blame to training.
2. Some of the results of training may not be evident for long periods of time. When training is directed at an immediate job problem, such as, firearms accuracy, immediate results can be expected. When training is a part of a large effort to transform the culture of an organization, as with cultural diversity training, results will emerge slowly over many years.
3. Training has a number of indirect benefits, which are differently valued by different people within the organization. People sent to training often are more motivated toward organizational goals because they have the sense that the organization cares about them. This may reduce turnover and contribute to the growth of human capital.
4. Training often has a history of abuse and absence of impact. Some of the more common problems are:
Brinkerhoff (p. 13) postulates three zones of training: before, during and after. Zone 1 represents those things done prior to training, determining needs, building readiness, gaining management support, identifying obstacles to performance and transfer of training, setting objectives, creating a project plan, and so forth. Zone 2 includes conducting the training, teaching trainees, leading training, measuring learning, and so forth. Zone 3 includes trainees attempting to use the training and supervisors interacting with newly trained employees, identifying and overcoming emerging barriers to utilization of training, identifying emerging training needs, and so forth.
In the next section, we will look at evaluation of training in law enforcement.
Evaluation in Law Enforcement
One of the areas of the legislatively required continuing education that has been evaluated recently is in the area of domestic or family violence. The Urban Institute evaluation of family violence training (Newmark, Harrell, & Adams, 1995) conducted under a federal grant included five states New York, Kentucky, Indiana, Massachusetts, and Texas because of the strength of their work in family violence training for law enforcement.
The Urban Institute study (Newmark, Harrell, & Adams, 1995) of domestic violence training noted in terms of evaluation issues that, "One difficulty we encountered in this study was the scarcity of outcome data from project records to support either: (1) that the training changed police knowledge, attitudes or behavior; or (2) that the training resulted in better services and protection for victims (p. 35)."
One of their findings from case study interviews was that officers' attitudes toward officer intervention in domestic violence cases and their knowledge of laws and policies was reported to be improved along with improved services. "Some respondents reported marked differences within the same county or jurisdiction between officers from agencies that participated in training and officers from agencies which had not received training (p. 63)."
Getting input from victim service units has been found to be a fruitful approach to evaluation. "Respondents from victim service agencies often stated that victims report more positive experiences with trained officers, greater willingness to call for police intervention, and appreciate provisions of progressive policies, such as mandatory arrest, because it takes the pressure off them to initiate legal proceedings and thereby risk retaliation from the abuser (Newmark, Harrell, & Adams, 1995, p. 63)."
Surveys of victims of domestic violence were used in the Urban Institute study. The Institute distributed copies of the ten page survey instrument to 128 programs with a total of 1,240 surveys in New York and Texas. The return rate for Texas was 221 out of 620 distributed in Texas. A copy of the survey is contained in Appendix C. The surveys were to provide data about satisfaction with law enforcement services. There were five key dimensions of services:
The study found that victims in Texas were more satisfied in the post-training period than had been found in an early satisfaction survey. The study design limited the conclusiveness of the results because so many factors could not be controlled.
The study did find areas that were rated particularly high by victims were seen as endorsements of many of the training lessons. They include:
Senate Interim Committee on Domestic Violence
A report from the Senate Interim Committee on Domestic Violence to the 74th Legislature in December 1994 addressed among other things law enforcement training regarding domestic violence. This Committee recommended the performance review by the Comptroller's Office that eventually resulted in this study.
The Committee received testimony that the quality of the training received was less than desirable, and that not all officers were in compliance. Some of the issues raised regarding law enforcement training were:
The legislature gave specific authority to discipline officers for failure to comply with the training requirements. The legislature also required the Commission to develop and mandate the minimum curriculum for the legislatively required domestic violence continuing education course.
Chapter Three:
Methodology
A survey was conducted primarily to answer the question about current practice. What is currently being done to evaluate the continuing education requirements?
The training providers that the Commission licenses or authorizes to deliver training and award continuing education credit engage in some level of evaluation. There are presently 105 licensed academies and 149 agreement trainers who provide training directly to the law enforcement officers of this State. Academies' responsibilities are described in Rule 211.65, and agreement trainers' responsibilities are described in Rule 215.66. These rules are contained in Appendix D.
Before proposing a new system of evaluation, it is crucial to understand what is currently being done by these training providers. The Commission obtained the services of an MBA student intern from the University of Texas School of Business. He did some of the preliminary research for the survey design. After review, a matrix design was used for the final questionnaire. A pilot test of ten training providers was conducted (from both academies and agreement trainers).
The instrument focused on several issues. The issues focused on included:
The survey was designed to obtain descriptive information about current practices and to elicit some opinions from the training providers in this area. The final survey as mailed is contained in Appendix E.
The surveys were mailed to the 254 training providers. A list of academy providers is contained in Appendix F. A list of the agreement trainers is contained in Appendix G.
One-hundred fifty-four (154) surveys were returned. The return rate was 61 percent. No follow-up contacts were made to increase the return rate because it was deemed to be a sufficiently representative sample with 75 (71 percent) of the academies and 78 (52 percent) of the agreement trainers returning their surveys. Appendix H displays those academies responding and those not responding. Appendix I displays those agreement trainers responding and those not responding.
In addition to the survey, contact was made with advocacy groups with an interest in continuing education programs for peace officers. Letters asking for input and ideas about the evaluation process were sent to the following groups:
No responses were received.
The data from the completed surveys were entered into data entry screen using SPSS software. Narrative data was entered into word processing software. The next section will report the results of the survey data.
Chapter Four:
Reporting of the Data
This section will report the results of the survey sent to the Commission approved law enforcement training providers. The descriptive results will be reported by training needs, courses taught, course evaluation, instructor evaluation, student evaluation, and recommendations. The complete results are shown in Appendix J.
Descriptive Results
Section I - Training Needs
Training providers reported that they determined training needs by a variety of methods. The four most used were: legislative requirements, specific departmental (section/division) requests, advisory board, and job analysis. Some them listed other methods which is reported in the comments section of Appendix J.
The methods were rated from one to five with one being the highest. The specific average ratings are in the following table.
Method of Determining Training End
Table Two
Average Method
2.0 Legislative requirements
2.2 Specific departmental (section/division)
requests
3.1 Advisory board
3.4 Other
3.6 Job analysis
4.5 Sources of funding (including fees)
4.6 Instructor availability
When the providers were asked which of the methods were most effective, they had a different response. The three highest rated were training advisory boards, specific departmental requests, and job analysis. Each training provider is required by Commission rule 215.67 to have an advisory board made up of law enforcement officials and public members. One-third of the members must be public members.
Rating of Most Effective Training Needs Assessment
Table Three
Number and Percentage Method
69 (44.8%) Advisory board
36 (23.4%) Specific departmental (section/division)
requests
10 (6.5%) Job analysis
2 (1.3%) Instructor availability
2 (1.3%) Legislative requirements
1 (0.6%) Sources of funding (including fees)
0 (0%) Other
When the providers were asked which of the methods were used to determine if a training need had been met, they had a different response. They question asked was, "Once the training needs have been identified, what methods are used to determine if the need has been satisfied?" The three highest responses were "All identified personnel have completed training" 76.6%, "Performance has improved" 50.0%, and "No demand for training" 20.1%.
Methods used to determine if identified training need has been met
Table Four
Number and Percentage Method
118 (76.6%) All identified personnel have completed
training
77 (50.0%) Performance has improved
31 (20.1%) No demand for training (not signing up for
course)
16 (10.4%) Other
When the providers were asked, "How do you determine future training needs?" Seventy-three percent responded by stating they did regularly scheduled needs assessments. The following table provides the results on this item.
How future training needs are determined
Table Five
Number and Percentage Method
113 (73.4%) Regularly schedule needs assessments
61 (39.6%) Projected new employees
46 (29.9%) Other
34 (22.1%) Projected promotions
26 (16.9%) Projected new equipment
The complete results for this section are shown in Appendix J.
Courses taught
The legislatively mandated courses are taught by the overwhelming majority of the training providers responding to the survey. Over ninety percent of the providers reported to be providing the Course 3232 and Course 3939.
Course 3232 is also available by correspondence by seven or 4.5% of the providers. Eleven or 7.1% of the providers reported having it available by some other manner. Course 3939 is also available by correspondence by five or 3.2% of the providers. Ten or 6.5% of the providers reported to have it available by some other manner. The complete results are shown in Appendix J.
Course evaluation
This section of the survey asked the respondent to rate the course. The item presented asked them to indicate "any methods your organization uses to evaluate the course itself."
The predominant choices for Course 3939 were "impressions and perceptions of the students" 77.9%, "formal critiques prepared by the students" 74.7%, and "impressions and perceptions of the instructors" 68.8%. The choices for the other courses were along the same lines. The complete results are shown in Appendix J.
Instructor evaluation
This section of the survey asked the respondent to rate the instructor. The item presented asked them to indicate "any methods your organization uses to evaluate the instructor(s)."
The predominant choices for Course 3939 were "formal critiques prepared by the students" 76.6%, "impressions and perceptions of the students" 72.1%, "formal review by management of student-prepared critiques" 51.9%, "formal review by management of lesson plans and/or class examinations" 48.7%, and "impressions and perceptions of other instructor(s)" 44.8%. The choices for the other courses were along the same lines. The complete results are shown in Appendix J.
Student evaluation
This section of the survey asked the respondent to rate the instructor. The item presented asked them to indicate "any methods your organization uses to evaluate the students."
The predominant choices for Course 3939 were "impressions and perceptions of other instructor(s)" 68.8%, "use of post-tests" 52.6%, "use of skill tests that physically demonstrate learned abilities" 29.2%, "use of pre-tests and post-tests" 27.3%, and "use of job performance reviews" 22.1%. The choices for the other courses were along the same lines. The complete results are shown in Appendix J.
Recommendations.
The recommendations section asked them to rate following evaluation methods for measuring quality and effectiveness of continuing education programs. The rating scale was as follows: 4 - very effective, 3 - effective, 2 - somewhat effective, and 1 - not effective. The following table shows the average rating on the methods listed. The highest rating was given to "other evaluation methods."
Most Effective Method of Evaluating Training
Table Six
Average Method
3.4 Other evaluation methods
3.0 Impressions and perceptions of the students
3.0 Formal critiques prepared by the students at the
completion of training
2.9 Use of job performance reviews
2.9 Impressions and perceptions of the instructors
2.9 Use of pre-tests and post-tests
2.7 Impressions and perceptions of academy
administration gained through observation of the
course
2.6 Use of post-tests only
2.0 Cost/benefit analysis
There were also three methods of funding questions asked. They were asked to agree or disagree with three listed methods. The following table contains those methods and their responses.
Funding Alternatives
Table Seven
Method Agree Disagree
Additional court costs (415.082) 83.8% 16.2%
Administrative processing fee based on course 24.0% 76.0%
load
Yearly fees for training licenses or agreements 18.2% 81.8%
They were also asked for ideas for any other methods of funding. Some of them include:
The complete results are shown in Appendix J.
Comparisons
It is useful to compare survey data to other sources to determine their validity, reliability, and feasibility. Industry can be a useful comparison for government because it is motivated by the bottom line, and usually will not engage in endeavors that are not feasible or that cost prohibitive. They are concerned with validity and reliability because they base business decisions off of the information that they collect.
The Industry Report 1996 from Training contains some interesting information about evaluation and other practices used in all sizes of companies. They conducted a stratified random sampling of the 146,837 companies in the Dun & Bradstreet database. Although benchmarks are usually chosen from those organizations that are leaders in their field, using the data might serve as a useful benchmark with which to compare our survey results. The following table shows rather large differences.
Comparison of Evaluation Approaches
Table Eight
Law Industry Evaluation Methods
Enforcement
74.7% 86.0% Trainee reactions
79.9% 71.0% Testing
22.1% 65.0% Behavior on the job
0.6% 49.0% Business results
The differences are
It is not surprising that these differences should exist. Business results and behavioral changes are harder to measure in law enforcement than in other forms of work. If we truly view training as being an investment, then it would seem to be important to try to determine what the return on that investment is.
In the next Chapter, we will try to summarize the results, along with alternatives, and recommendations.
Chapter Five:
Summary, Alternatives, and Recommendations
This chapter will provide some summary comments, a discussion of alternatives, and recommendations. Up to this point, we have looked at what evaluation is being done in the general field of training and what is being done in Texas law enforcement training. We will next discuss why evaluation of this training should be done.
Why should it be done?
There several reasons for evaluating the continuing education programs in a thorough and professional manner.
There are tremendous costs associated with such a broad based continuing education program when so many people are involved. As Spencer (1986) has noted 85 percent of training cost is the student's time (p. 69). As noted in the Introduction the estimated cost for this program is over $28 million dollars. This is an amount 16 times larger than the Commission's budget.
There is an economy of scale in having one entity conducting evaluation as opposed to 254 training providers doing a different approach to evaluation with data that cannot be aggregated for any type of meta-analysis of the needs and successes of the programs. Without this type of comprehensive data, the legislature is left to make many critical policy decisions based upon anecdotal evidence and information.
The quality and effectiveness of the training could be most economically increased by providing sufficient funds to the Commission for staff to spend the time to develop a comprehensive needs assessment, program delivery, and evaluation system with the ancillary resources that are needed.
Other contributions of evaluation studies
Evaluation of training studies can make other related contributions to law enforcement improvements. Some of these contributions are: identify changes to training programs to make them more effective, identify other needs that might assist in improving law enforcement services, identify when a training program is no longer needed or should be replaced, or identify other changes that might effect greater gains in performance. For instance the Urban Institute study (Newmark, Harrell, & Adams, 1995) of domestic violence training noted that instead of repetitive comprehensive training that, "Brief refresher training sessions, such as roll call training, can be quite helpful in keeping officers informed of new developments and updates of training materials. They can also be useful in reinforcing key policy elements and as a tool in policy implementation and supervision (p. 65)." This suggests that instead of repetitive training that bulletins or newsletters might be as useful in updating officers, as a full blown formal training class.
What should be done?
There are alternatives which the legislature might consider in improving the quality and effectiveness of the continuing education program for law enforcement. The next section will discuss them.
Alternatives
There are several alternative approaches to addressing the "evaluation problem." They are: use the classic Kirkpatrick Model, use Brinkerhoff's three zones, develop cost benefit analysis studies, develop a classic experimental model using control and experimental groups, surveys of victims of crime or customers of law enforcement services similar to the Urban Institute study, or do some combination of these approaches.
Using the Kirkpatrick Model
Kirkpatrick's four levels of evaluation is a place to start when you have a course already up a running and you want to justify the training.
LEVEL I: Use a course feedback form to see if participants liked the training (Level I); include the questions, "Will you be able to apply these skills on the job? Why or why not? How will these new skills impact your job performance? What specific things will you do differently as a result of this training?"
LEVEL II: Do performance testing (not just knowledge testing) throughout the training (Level II) to make sure they can DO the skills. (Of course, this requires that we come up with performance criteria....)
LEVEL III: Follow up the training after a month or so to determine what skills and knowledge from the training "stuck" and are now being applied on the job (Level III). Talk to students; more importantly, talk to their managers. Ask them what students have done differently and what that difference has meant in terms of job performance (reduced errors, increased productivity, increased quality, etc.)
LEVEL IV: Use Level III to make the link to Level IV evaluation (determining the impact of the application of the new skills and knowledge). You can collect anecdotal information from managers by asking, "You said that the newly trained people are serving the community better. How much better? What impact does this have on arrests? On community satisfaction surveys?" You can ask for data points (audit results, arrest measures, etc.) that may provide additional insight into the business impact.
Brinkerhoff's Three Zone Approach
Brinkerhoff's three zones is a simpler version of the Kirkpatrick model. It focuses on before, during, and after. Brinkerhoff identified some questions for each Zone.
Zone 1.
Zone 2.
Zone 3.
Survey Research
Survey research of victims of crime and law enforcement service can add a useful perspective and should be a part of either of the above approaches of either Kirkpatrick's level four or Brinkerhoff's zone three evaluation times. A copy of the instrument used by the Urban Institute study (Newmark, Harrell, & Adams, 1995) is contained in Appendix C.
Cost-Benefit Analysis
This is a most difficult type of evaluation, but it can be extremely valuable for policy-makers in decision-making. A study funded by the National Institute of Justice found that the cost of one child being sexually abused was $125,000 (Miller, Cohen, & Wiersema, 1996, p.16). If training can reduce the number of victims by even a few, then it will be worth a significant training investment. Table Nine below summarizes their findings.
Cost for Victimization
Table Nine
Per Victim
Crime without risk of death with risk of death
Child abuse: sexual $125,000 *
Rape & sexual assault $109,000 $110,000
(excluding child abuse)
Child abuse: physical $77,000 *
Child abuse (all types) $70,000 $74,000
Arson $38,000 $54,000
Child abuse: emotional $30,000 *
Drunk driving $18,000 $26,000
Assault or attempt $12,000 $31,000
Assault (any) $14,000 $16,000
Robbery or attempt $10,000 $16,000
Motor vehicle theft $4,000 $4,000
Burglary $1,600 $1,700
Larceny $400 $400
*Deaths due to child abuse are not categorized by type of child abuse (e.g., sexual, physical, or emotional). Thus, no estimates are provided that include the risk of death. However, a combined child abuse category is included in this table, which includes the risk of death estimate.
A comparison of the cost of conducting the training and the officer's time in attending the training to the benefits of preventing one case of child sexual abuse can be made. A difficult measure is how do you determine when a case has been prevented. One option is to look at crime statistics prior to the training and after the training. There are a number of problems with this because in some cases increased awareness of a problem results in a higher reporting of it. Crime victimization surveys can lessen that possibility somewhat. Another option is to determine if arrest, prosecution, and incarceration rates for trained officers increases.
Another option more economical model is to have officers rate after they have returned to the job any increases in effectiveness and efficiency as a result of the training. For example, if an officer spends ten percent of his time interacting with people of a different cultural group and he notes a twenty percent increase in his effectiveness in dealing with situations that involve members of these groups because of the training, then a benefit can be calculated based upon his salary. In this example a benefit of $500 can be identified.
Example: .10 X .20 X $25,000 Salary = $500 benefit.
In actual application, the officer's cost is much higher because of support and other costs. This would have to be determined by research. This is based upon a salary for one year although the benefits of training usually will last for several years.
Distance Education
One of the problematic areas in terms of evaluation is Distance Education. The Standards for Evaluation of Distance Education should follow the Guiding Principles for Distance Learning in a Learning Society and the Distance Learning Evaluation Guide of the American Council on Education should be adopted as guidelines for evaluation of these programs. A comprehensive comparative study evaluating the various modalities used in these programs should be conducted.
Other approaches
Included within the courses was general rating instruments which were to be used by the training providers. This information could be aggregated into an evaluation data collection survey. This could be refined and expanded in future years. A copy of these are contained in Appendix K.
How should it be done?
The Urban Institute evaluation of family violence training (Newmark, Harrell, & Adams, 1995) advocate site visits, in part, because site visits allow for interviews to be conducted with key personnel which allows for the acquisition of much rich and detailed information. Sufficient site visits by the Commission can only be attained by additional staffing and travel funds.
Quality and Effectiveness
Field evaluators should look for a number of factors related to the quality and effectiveness of the training. Typically formal evaluations are summative by nature withholding findings until the study is complete, so as to not interfere with the study. These evaluations should be formative evaluations which are consultative in nature with suggestions for improvements made during the process.(Borich & Jemelka, 1982) Some of the factors that they should look for are:
Quality -
Effectiveness -
Agencies that are provided training by selected providers will be surveyed to determine satisfaction with training. Officers that have been provided training by selected providers will be surveyed to determine satisfaction with training. Surveys will be mailed out with stamped self addressed return envelopes, with a deadline, phone call follow-ups will be made for non returns.
Plan for Implementation
The evaluation process will begin as soon as approved by the legislature and the funding required is secured. Hiring of additional staff can be implemented by September 1, 1997. Initial development of effective needs assessment and evaluation plans can be completed by January 1, 1998.
Budget and Funding Sources
Budgeting
For the Commission to execute any of these models will take a team to work with each of the training providers and agencies. With over 250 training providers and 2500 agencies, five personnel could be assigned to evaluate the programs at the delivery level and the agencies at the implementation level. Each person would have a caseload of 50 training providers and 500 agencies. With five new personnel in the field dedicated to evaluation, a significant sampling of training impact can be accomplished to ensure the quality and effectiveness of the training. An additional person needs to be assigned to conduct needs assessments, curriculum development projects, and follow-up survey research including cost benefit analysis.
The cost for these six people including benefits would be $259,866. Operational costs for travel, supplies, and so forth would be about $12,000 a year. Additional costs for capitol expenditures for computers and furniture would be a one time cost of approximately $20,000.
Funding
The amount of funding required will be determined by: the number of providers to be evaluated, the number of agencies to be surveyed, the number of officers to be surveyed, plus the amount of detail in the evaluation. There are several alternatives for funding.
Funds from the Continuing education funds
There is a $2 surcharge on all fines collected in the State. A percentage of those funds could be allocated to the Commission for implementation of evaluation programs. Ten per cent of the funds collected for this mandated training is a reasonable amount for the Commission to develop and evaluate this training.
Funds from Fine Surcharge
The Commission is currently funded by a $1 surcharge on fines collected in the State. It currently receives about 47 percent of those funds. If the remaining funds were dedicated for the implementation of the evaluation program, there would be more than sufficient funds for this program. Eighty-four percent of the training providers endorsed this method.
Fees from provider licensing
There are 105 academies and 150 agreement trainers in Texas. Fees for this could be collected with the funds earmarked for the implementation of the evaluation program. Eighteen percent of the training providers endorsed this method.
Fees for continuing education courses reported
The Commission could be required to collect a fee for each person who takes a course. We currently receive, store, and record that information without charge. Twenty-Four percent of the training providers endorsed this method
Fees for licensing and certificates
The Commission currently does not charge for licensing. It does charge a fee for some of the proficiency certificates. A fee for these services could be charged with the money earmarked for a comprehensive program development, delivery, and evaluation system. A twenty dollar fee for a four year licensing period would cover the administrative costs and the costs of this program. It would also make it easier to ensure compliance with the continuing education requirements. This could be done by requiring compliance with the legislatively required continuing education program as a condition for renewal.
Any of the above methods should be able to provide the necessary funding to improve the quality and effectiveness of the continuing education programs required under 415.034 Government Code. The Commission is ready to provide additional information upon the request of any member of the legislature.
References
American Council on Education (1996). Guiding Principles for Distance Learning in a Learning Society. Washington, D. C.: American Council on Education.
American Council on Education (1996). Distance Learning Evaluation Guide. Washington, D. C.: American Council on Education.
Borich, G. D., & Jemelka, R. P. (1982). Programs and systems: An evaluation perspective. New York: Academic Press.
Brinkerhoff, R. O. (1989). Using evaluation to transform training. Robert O. Brinkerhoff (ed.). Evaluating training programs in business and industry. New Directions for Program Evaluation, no. 44. San Francisco: Jossey-Bass.
Caffarella, R. S. (1988). Program development and evaluation resource book for trainers. New York: John Wiley & Sons.
Cascio, W. F. (1987). Costing human resources: The financial impact of behavior in organizations 2nd Ed. Boston: PWS--Kent Publishing.
Erickson, P. R. (January, 1990). Evaluating training results. Training and Development Journal. pp. 57-59.
Moncrief, M., Brown, J. E., Shapiro, F., West, R., & Zaffirini, J. (1994). Senate Interim Committee on Domestic Violence: Report to the 74th Legislature. Austin: Senate Interim Committee on Domestic Violence.
Geber, B. (March, 1995). Does training make a difference?: Prove it! Training.
Goldstein, I. I. (1993). Training in organizations: Needs assessment, development, and evaluation, 3rd Edition. Pacific Grove, CA: Brooks/Cole Publishing.
Miller, T. R., Cohen, M. A., & Wiersema, B. (1996). Victim costs and consequences: A new look. Washington, D. C.: National Institute of Justice.
Miller, T. R., Cohen, M. A., & Wiersema, B. (January, 1996). The extent and costs of crime victimization: A new look. National Institute of Justice Research Preview. Washington, D. C.: National Institute of Justice.
Newmark, L., Harrell, A., & Adams, B. (1995). Evaluation of police training conducted under the family violence prevention and services act. Washington, D.C.: The Urban Institute.
Newmark, L., Harrell, A., & Adams, B. (November, 1995). Evaluation of family violence training programs. National Institute of Justice Research Preview. Washington, D. C.: National Institute of Justice.
Robinson, D. G., & Robinson, J. (August, 1989). Training for Impact. Training and Development Journal. pp. 34-42.
Schmidt, F. L., & Hunter, J. E. (1986). Employment testing: Old theories and new research findings. In S. L. Rynes & g. T. Milkovich (Eds.), Current issues in human resource management: Commentary and readings, Business Publications.
Sharp, J. (1995). Advisory: Criminal justice education in Texas. Austin: Texas Comptroller of Public Accounts.
Shinberg, B. (1983). What is competence? How can it be Assessed? In Stern, M. R., ed., Power and conflict in continuing professional education. Belmont, CA: Wadsworth Publishing Company and the National University Continuing Education Association.
Spencer, L. M., Jr. (1986). Calculating human resource costs and benefits: Cutting costs and improving productivity. New York: John Wiley & Sons.
Stern, M. R., ed. (1983). Power and conflict in continuing professional education. Belmont, CA: Wadsworth Publishing Company and the National University Continuing Education Association.
Training (October 1996). Industry report 1996. Training. pp. 37-79.
Appendix A
415.034 Government Code
Sec. 415.034. CONTINUING EDUCATION.
(a) The commission shall recognize, prepare, or administer continuing
education programs for officers and county jailers.
(b) The commission shall require a state, county, special district, or
municipal agency that appoints or employs peace officers to provide each
peace officer with a training program every 24 months. The course may
not exceed 40 hours. Not less than 20 hours of the instruction must be
on topics selected by the agency.
(c) The course provided under Subsection (b) must:
(1) be approved by the commission;
(2) include education and training in:
(A) civil rights, racial sensitivity, and cultural
diversity; and
(B) unless determined by the agency head to be
inconsistent with the officer's assigned duties:
(i) the recognition of cases that involve child
abuse, child neglect, family violence, and sexual assault, and
(ii) issues concerning sex offender characteristics;
and
(3) include other education and training only if determined by
the agency head to be consistent with the officer's assigned duties.
(d) A peace officer appointed to the officer's first supervisory
position must receive in-service training on supervision as part of the
course provided under Subsection (b) during the 24-month period after
the date of that appointment.
(e) An honorably retired commissioned officer of the Department of
Public Safety who is a special ranger under Section 411.023 may not be
required to undergo training under Subsection (b).
(f) The commission may require a state, county, special district, or
municipal agency that appoints or employs a reserve law enforcement
officer, county jailer, or public security officer to provide each of
those individuals with education and training in civil rights, racial
sensitivity, and cultural diversity every 24 months.
The version of text set out below as §415.034 was amended by Acts 1995,
74th Leg., Chapter 538, §2 (Senate Bill 1337)
Sec. 415.034. CONTINUING EDUCATION.
(a) The commission shall recognize, prepare, or administer continuing
education programs for officers and county jailers.
(b) The commission shall require a state, county, special district, or
municipal agency that appoints or employs peace officers to provide each
peace officer with a training program every 24 months. The course must:
(1) be approved by the commission;
(2) contain curricula which incorporate the learning objectives
developed by the commission; and
(3) include education and training in:
(A) civil rights, racial sensitivity, and cultural
diversity; and
(B) unless determined by the agency head to be
inconsistent with the officer's assigned duties, the recognition of
cases that involve the following:
(i) child abuse;
(ii) child neglect;
(iii) family violence; and
(iv) sexual assault.
(c) The course provided under Subsection (b);
(1) may not exceed 40 hours; and
(2) may include instructional materials developed by the agency
or its trainers or entities having training agreements with the
commission in addition to materials included in curricula developed by
the commission.
(d) A peace officer appointed to the officer's first supervisory
position must receive in-service training on supervision as part of the
course provided under Subsection (b) during the 24-month period after
the date of that appointment.
(e) An honorably retired commissioned officer of the Department of
Public Safety who is a special ranger under Section 411.023 may not be
required to undergo training under Subsection (b).
(f) The commission may require a state, county, special district, or
municipal agency that appoints or employs a reserve law enforcement
officer, county jailer, or public security officer to provide each of
those individuals with education and training in civil rights, racial
sensitivity, and cultural diversity every 24 months.
(g) The commission may suspend the license of a peace officer if the
peace officer fails to complete a training course at least once in every
24-month period. The commission shall adopt rules under which the
training course requirement provided by this subsection may be waived
when mitigating circumstances exist.
(h) The commission by rule shall adopt procedures:
(1) to ensure the timely and accurate reporting by agencies and
peace officers of information related to training programs offered under
this section, which procedures shall provide for the creation of
training records for individual peace officers;
(2) to provide adequate notice to agencies and peace officers of
impending noncompliance with the training requirements of this section
so that the agencies and peace officers may comply within the 24-month
period;
(3) to require agencies to report in a timely manner the reasons
that a peace officer is in noncompliance after receiving notice by the
commission of the peace officer's noncompliance; and
(4) to provide that, following the commission's receipt of an
agency's report under Subdivision (3) or on a determination by the
commission that the agency has failed to report in a timely manner, a
hearing consistent with Section 415.060(b) shall be held when the peace
officer claims that:
(A) mitigating circumstances exist; or
(B) the peace officer failed to complete the required
training because the officer's employing agency did not provide an
adequate opportunity for the officer to attend the required training
course.
The version of text set out below as §415.034 was amended by Acts 1995,
74th Leg., Chapter 562, §2 (Senate Bill 225)
Sec. 415.034. CONTINUING EDUCATION.
(a) The commission shall recognize, prepare, or administer continuing
education programs for officers and county jailers.
(b) The commission shall require a state, county, special district, or
municipal agency that appoints or employs peace officers to provide each
peace officer with a training program every 24 months. The course must:
(1) be approved by the commission; and
(2) include education and training in:
(A) civil rights, racial sensitivity, and cultural
diversity; and
(B) unless determined by the agency head to be
inconsistent with the officer's assigned duties, the recognition and
documentation of cases that involve the following:
(i) child abuse;
(ii) child neglect;
(iii) family violence; and
(iv) sexual assault.
(c) The course provided under Subsection (b) may not exceed 40 hours.
(d) A peace officer appointed to the officer's first supervisory
position must receive in-service training on supervision as part of the
course provided under Subsection (b) during the 24-month period after
the date of that appointment.
(e) An honorably retired commissioned officer of the Department of
Public Safety who is a special ranger under Section 411.023 may not be
required to undergo training under Subsection (b).
(f) The commission may require a state, county, special district, or
municipal agency that appoints or employs a reserve law enforcement
officer, county jailer, or public security officer to provide each of
those individuals with education and training in civil rights, racial
sensitivity, and cultural diversity every 24 months.
(g) Training in documentation of cases required by Subsection (b)
shall include instruction in:
(1) making a written account of the extent of injuries sustained
by the victim of an alleged offense;
(2) recording by photograph or videotape the area in which an
alleged offense occurred and any injuries sustained by the victim; and
(3) recognizing and recording any statement of a victim that may
be admissible as evidence in a future proceeding concerning the matter
about which the statement was made.
The version of text set out below as §415.034 was amended by Acts 1995,
74th Leg., Chapter 585, §2 (Senate Bill 1135)
Sec. 415.034. CONTINUING EDUCATION.
(a) The commission shall recognize, prepare, or administer continuing
education programs for officers and county jailers.
(b) The commission shall require a state, county, special district, or
municipal agency that appoints or employs peace officers to provide each
peace officer with a training program every 24 months. The course may
not exceed 40 hours. Not less than 20 hours of the instruction must be
on topics selected by the agency
(c) The course provided under Subsection (b) must:
(1) be approved by the commission;
(2) include education and training in:
(A) civil rights, racial sensitivity, and cultural
diversity; and
(B) unless determined by the agency head to be
inconsistent with the officer's assigned duties, the recognition of
cases that involve the following:
(i) child abuse;
(ii) child neglect;
(iii) family violence; and
(iv) sexual assault; and
(3) include other education and training only if determined by
the agency head to be consistent with the officer's assigned duties.
(d) A peace officer appointed to the officer's first supervisory
position must receive in-service training on supervision as part of the
course provided under Subsection (b) during the 24-month period after
the date of that appointment.
(e) An honorably retired commissioned officer of the Department of
Public Safety who is a special ranger under Section 411.023 may not be
required to undergo training under Subsection (b).
(f) The commission may require a state, county, special district, or
municipal agency that appoints or employs a reserve law enforcement
officer, county jailer, or public security officer to provide each of
those individuals with education and training in civil rights, racial
sensitivity, and cultural diversity every 24 months.
Appendix B
TCLEOSE Rule 217.9§217.9. Continuing Education for License Holders.
(a) Each agency that appoints or employs peace officers, reserve law enforcement officers, county jailers, or public security officers shall provide each peace officer, reserve law enforcement officer, county jailer, or public security officer it appoints or employs a continuing education program at least once every 24 months.
(b) The commission shall develop curricula with learning objectives for the statutorily required continuing education programs that include the following topics:
(1) civil rights, racial sensitivity, and cultural diversity;
(2) the recognition and documentation of cases that involve child abuse, child neglect, family violence, sexual assault, and issues concerning sex offender characteristics; and
(3) supervision.
(c) The required continuing education program for peace officers shall consist of 40 hours of training. The required program as outlined in (b) above shall contain no more than 20 hours of curricula and learning objectives. The remaining hours may consist of additional objectives and materials selected or developed by the appointing or employing agency. The additional topic or topics selected by the agency should be consistent with the peace officer's assigned duties. This rule does not limit the number of hours of continuing education an agency may provide to each peace officer, reserve law enforcement officer, county jailer, or public security officer it appoints or employs.
(d) As part of the 20 hours of required training which must include the curricula and learning objectives developed by the commission:
(1) each peace officer, reserve law enforcement officer, county jailer, or public security officer shall complete the course of training developed by the commission in civil rights, racial sensitivity, and cultural diversity during each 24 month period;
(2) each peace officer shall complete the course of training developed by the commission in the recognition and documentation of cases that involve child abuse, child neglect, family violence, sexual assault, and issues concerning sex offender characteristics during each 24 month period. If an agency chief administrator determines these subjects to be inconsistent with the peace officer's assigned duties, the chief administrator may substitute other training determined to be consistent with the officer's assigned duties;
(3) each constable and deputy constable shall also complete a 20 hour course of training in civil process during each 24 month period. The commission may waive the requirement for civil process training if the constable requests a waiver, by written certification, because of hardship and the commission determines that a hardship exists; and
(4) each peace officer appointed to their first supervisory position must complete the course of training developed by the commission in supervision as part of the required training within 24 months following the date of appointment as a supervisor.
(e) The commission shall provide notice to agencies and officers of impending non-compliance with the continuing education requirements. Such notice will be provided not later than six months prior to the expiration of the 24 month period.
(f) The commission may suspend a peace officer's license for failure to complete the required continuing education program at least once every 24 months.
(g) Each peace officer who is currently licensed and reported to the commission as appointed or employed by an agency on or before September 1, 1995 shall complete the continuing education program required under this section before September 1, 1997. Any officer licensed or appointed after September 1, 1995 shall have 24 calendar months from the date of initial licensing to complete the continuing education program required under this section.
(h) Subsequent 24 month periods for continuing education shall begin on the last day of the prior period and end 24 months thereafter.
(i) The effective date of this section is April 15, 1996.
Appendix C
Urban Institute Survey
Appendix D
Training Provider Rules
§211.65. Academy Licensing, Operations, and Evaluation.
(a) The commission may issue an academy license to an academy that is operated by or for the state or any political subdivision of the state for the specific purpose of providing law enforcement and/or corrections training.
(b) Prior to being issued an academy license, an academy must pass an inspection of its facilities and instructional materials and must submit for commission approval:
(1) a completed, written application on a commission form that is signed by the chief administrator or head of the organization exercising administrative control over the academy;
(2) a resolution of support from the governing body of the sponsoring organization;
(3) the formal name of the academy, which must not misrepresent the status of the academy or be confusing to law enforcement or to the public;
(4) a proposed startup and operational budget and a proposed course schedule to show that training will be conducted on a continuing basis;
(5) evidence that an advisory board has already been appointed as provided by Section 415.031 of the Government Code, including a resume for each board member;
(6) any advisory board minutes necessary to show the decisions which have been made by that board in all areas required by the commission;
(7) the name, social security number and resume of the proposed training coordinator and any academy staff instructors, and a list of instructors who are scheduled to teach the submitted proposed course schedule;
(8) evidence that the academy will be, based on the characteristics of the sponsoring organization, at least one of the following:
(A) an agency academy, conducted by a law enforcement agency that has at least 50 full-time paid peace officers and/or county jailers under current appointment;
(B) a college academy, conducted by an institution coordinated by the Texas Higher Education Coordinating Board; or
(C) a regional academy, conducted or sponsored by a regional planning commission or council of governments (COG) board;
(i) The commission will only issue one regional academy license within each regional planning commission or council of governments area at any one time.
(ii) To be or remain a regional academy, that particular academy must substantially meet the training needs of all current or prospective license holders who reside in that region and do not attend an agency academy or college academy.
(9) certification that the academy meets the requirements of the Americans with Disabilities Act (ADA), to which its entity is subject, and as those requirements apply to the academy's function (including course materials, course presentation, and facilities). The certification will represent that the academy will maintain this compliance during the term of the license;
(10) the physical location and a description of the proposed training facility; and
(11) a comprehensive training needs assessment justifying the need for an additional academy in the regional planning commission or council of governments area in which the proposed academy is located. The needs assessment must include as a minimum:
(A) a description of whom the academy will serve, including the identity of each law enforcement agency the academy expects to serve, the number of officers the academy expects to train annually from each agency, and the basis for the academy's expectations;
(B) a schedule of tuition and fees, if any, that will be charged;
(C) a description of existing law enforcement training programs in the proposed service area and evidence justifying the need for an additional academy;
(D) the number and types of courses that will be offered;
(E) what specific training need(s) are not currently being provided by licensed academies in the regional planning commission or council of governments area; and
(F) applicant must show proof of notification by certified mail to all licensed academies within the regional planning commission or council of governments area of their intent to apply for academy license and what specific training needs are not currently being met within the region.
(c) The pre-licensing inspection of the academy's facilities and instructional material shall be conducted by the commission. The commission may appoint an inspection team composed of persons with experience in the field of law enforcement education and training.
(d) To pass a pre-licensing inspection, an academy must have and maintain :
(1) a classroom that is sufficiently air-conditioned and heated, well lit, free of noise and other unreasonable distractions, and of sufficient size for the number of students to be served;
(2) instructors and adequate instructional resources to conduct effective training;
(3) adequate and convenient restrooms, breakroom, and parking area;
(4) adequate and convenient law enforcement reference library for student and staff use; and
(5) must have access to an all-weather accessible firing range suitable for the course of fire required in the basic peace officer course with safety rules clearly posted, and adequate restrooms and first aid equipment on the premises.
(e) The chief administrator or head of the organization exercising administrative control of the academy and the proposed training coordinator must appear before the commissioners to respond to any questions prior to any action being taken on the application.
(f) A training coordinator must hold a valid instructor license and must be paid and assigned on a full-time basis. To be considered paid and assigned on a full-time basis, the training coordinator must be a full-time paid employee of the agency/institution. If the coordinator has additional job responsibilities other than the academy, one or more professional personnel must be assigned full-time as academy training/educational staff.
(g) The training coordinator of an academy must:
(1) prepare, maintain, and submit the following reports within the time frame specified:
(A) reports of training - to be submitted within 30 days of completion of each course;
(B) quarterly training summaries - to be submitted within 10 days of the close of the reporting period;
(C) advisory board minutes- to be submitted within 10 days of board approval;
(D) quarterly training calendars - to be submitted no later than 10 days prior to the beginning of each calendar quarter; and
(E) any other reports or records as requested by the commission;
(2) receive all commission notices on behalf of the academy and forward each notice to the appointing authority;
(3) be responsible for the administration and conduct of each course, including those conducted at ancillary sites and specifically:
(A) appointing and supervising qualified instructors;
(B) maintaining course schedules and course files;
(C) securing and maintaining all facilities necessary to meet the inspection standards of this section;
(D) enforcing all admission, attendance, retention, and other standards set by the advisory board;
(E) distributing learning objectives to all students and insuring that all learning objectives are taught, that all training is effective, and that no required instruction periods are consumed by matters that are frivolous or unrelated to the scheduled training;
(F) controlling the discipline and demeanor of each student and instructor during class; and
(G) proctoring or supervising all examinations to insure fair, honest results;
(4) attend or have his or her designee attend each academy coordinator's workshop conducted by the commission.
(h) The chief administrator of the academy or the sponsoring agency must report in writing to the commission within 10 days:
(1) any change in training coordinator;
(2) any substantial failure to meet the inspection standards;
(3) any rule violation by it or by its training coordinator, instructors, or advisory board;
(4) when non-compliance with ADA requirements is discovered; or
(5) any change in academy name, physical location, mailing address or telephone number.
(i) The commissioners may cancel an academy license if it was issued in error or based on false or incorrect information.
(j) The commissioners may suspend an academy license, or the executive director or his designee may issue a written reprimand to the sponsoring agency, if:
(1) the academy or the sponsoring agency fails to comply with a commission rule or law; or
(2) the academy receives a rating of "below expectations" or "unacceptable" on any component in the academy evaluation process. If the academy receives a rating of "below expectations" or "unacceptable", the chief administrator of the academy or the sponsoring agency must report to the commission in writing within 30 days what steps have been taken to correct deficiencies and on what date they expect to be in compliance.
(k) The commissioners may revoke an academy license if:
(1) it has received two or more suspensions and/or reprimands within a twelve month period;
(2) its training coordinator intentionally or knowingly submits a falsified document or a false written statement or representation to the commission ; or
(3) it has received a rating of "below expectations" or "unacceptable" on the same component in the academy evaluation process on two successive evaluations.
(l) An academy may voluntarily surrender its license at any time for any reason. To voluntarily surrender its license, an academy's chief administrator must send written notice, accompanied by the license, to the executive director. The license is surrendered effective immediately upon receipt by the executive director.
(m) The commission will award training credit for any course conducted by a licensed academy as provided by commission rules unless:
(1) the course is not taught as required by commission rules and the advisory board; or
(2) the training is not related to a commission license; or
(3) the advisory board, the academy, the training coordinator, the course coordinator, or the instructor substantially failed to discharge any responsibility required by commission rule.
(n) If the position of training coordinator becomes vacant, the commission may at the discretion of the executive director and upon petition of the chief administrator of the academy or sponsoring agency waive the requirement for a full-time paid and assigned coordinator for a period not to exceed six months.
(o) After licensing, the academy may be inspected by the commission at any time and will be evaluated periodically, as determined by the commission. The inspection may be accomplished by commission staff or by training professionals selected and trained by commission staff.
(1) The evaluation program consists of four components:
(A) Analysis -- a rating will be assigned based on the overall class average on all licensing exams taken by students of the academy. If an academy does not offer these types of courses this component will not be applicable;
(B) Compliance -- commission records will be reviewed to determine if an academy is in compliance with the commission rules and regulations;
(C) Assessment -- an on-site inspection of all facilities, equipment, and instructional materials as well as an inspection of the academy's management documentation will be conducted; and
(D) Appraisal -- a classroom observation and instructor evaluation will be conducted.
(2) An overall evaluation of the academy will then be made based upon the ratings of the four components. The academy training coordinator and chief administrator will be notified of the results and any recommendations or action to be taken.
(p) The effective date of this section is February 1, 1989. The effective date of subsections (a)-(o) of this section as amended is September 1, 1994.
§215.66. Agreement Training.
(a) The commission may, in the discretion of the executive director, enter into an agreement with a law enforcement agency, a law enforcement association, or alternative delivery trainer to conduct training for license holders.
(b) Any such agreement is limited to those terms expressly included in the agreement or incorporated by reference and must be dated and:
(1) in writing on a commission form;
(2) signed by the executive director;
(3) signed by the chief administrator or head of the sponsoring organization; and
(4) signed by the training coordinator responsible for the administration of that training.
(c) An agreement may approve a specific course(s) and the number of times it will be offered. These contracts are perpetual but may be terminated for cause within ten days by written notice on the part of either party to the contract. An agreement may incorporate by reference a law, rule, or any other document. However, any waiver, exception, or deletion must be express.
(d) The executive director may terminate an agreement if no training is conducted within each calendar year unless the chief administrator has petitioned the executive director for a waiver, and the waiver has been granted. The executive director may suspend an agreement, until compliance, for any violation of its terms or of any commission rule or law. Any party may terminate upon written notice to all other parties, received by either the executive director, the coordinator, or any other named person or office.
(e) The agreeing agency, association, or alternative delivery trainer must:
(1) provide a comprehensive needs assessment to the executive director justifying the need for an agreement. The needs assessment must include at a minimum:
(A) the names of the licensed academies located in the council of governments or regional planning commission area of the requesting party;
(B) a description of the existing law enforcement training programs in the area;
(C) what specific training need(s) are to be addressed by the proposed agreement contract.
(D) the number and types of courses that will be offered during the first quarter of the executed contract;
(2) appoint and maintain an advisory board as required by law and rule;
(3) follow the current requirements set by its advisory board;
(4) select a training facility that meets all academy inspection requirements, if applicable;
(5) select any instructional material, equipment, or resources necessary for the course;
(6) forward for approval, upon the executive director's request, at least one copy of the learning objectives of each course covered by the agreement;
(7) appoint and maintain the appointment of a qualified training coordinator;
(8) insure the training coordinator discharges any responsibilities required by law, rule, or agreement;
(9) select and monitor the performance of qualified instructors;
(10) admit any license holder subject to any reasonable limitations or preferences required by the advisory board;
(11) insure effective training and distribute learning objectives to each student before the course is taught;
(12) teach or insure that each course is taught in accordance with the instructor guide and/or learning objectives provided or approved by the commission;
(13) keep records of all agreement training for at least five years; and
(14) proctor any required
examination and insure fair, honest results.
(f) Unless expressly waived by the agreement:
(1) an advisory board for agreement training must discharge the responsibilities of such boards as required by law or rule; and
(2) a training coordinator for an agreement must discharge the same responsibilities as an academy training coordinator and must hold a valid instructor license.
(g) By entering into any such agreement, the commission preapproves specific training which will be fully credited by the commission to each student as basic or in-service training or to the agency as in-service training provided by that agency, unless:
(1) the training was not conducted in compliance with the agreement; or
(2) the advisory board, training coordinator, or instructor substantially failed to discharge any responsibility required by rule.
(h) Once the agreement has been executed, the agreement trainer may be evaluated periodically by the commission as determined by the executive director. The evaluation may be accomplished by commission staff or by training professionals selected and trained by commission staff.
(i) The effective date of this section is September 1, 1994.
Appendix E
Survey Instrument
Texas Commission on Law Enforcement
______________________________
QUESTIONNAIRE ON CONTINUING EDUCATION
Name of Training Provider __________________________________________________
Mailing Address __________________________________________________________
Street City State Zip Code
Telephone Number (____)_______________ Fax Number (____)_____________
Name of Training Coordinator _______________________________________________
Name of Person Who Completed Questionnaire _________________________________
Total Number of All Instructors (including Full-time and Part-time) _______
Type of Training Provider: Licensed Academy
Agreement Trainer
Types of law enforcement agencies for which you provide continuing education training
(Please check all that are applicable):
Airport Police Departments
College and University Police Departments
Constable's Offices
County Attorney's Offices
District Attorney's Offices
Hospital Police Departments
Municipal Police Departments
Park Police Departments
School District Police Departments
Sheriff's Departments
State Agencies
Other - Please list types:
________________________________________
________________________________________
________________________________________
SECTION I
Training Needs
Every training provider has a method for identifying what type of training is needed for continuing education. Please indicate how you determine this need. (please rank these options using
1 = highest priority)
___ Advisory Board
___ Specific departmental (section/division) requests
___ Job Analysis
___ Instructor availability
___ Sources of funding (including fees)
___ Legislative requirements
___ Other (please list)
________________________________________________________________________
________________________________________________________________________
Which is most effective? ____________________________________________________
Once the training needs have been identified, what methods are used to determine if the need has been satisfied?
All identified personnel have completed training
No demand for training (not signing up for course)
Performance has improved
Other (please list)
________________________________________________________________________
________________________________________________________________________
How do you determine future training needs
Regularly Scheduled Needs Assessments
Projected new employees
Projected new equipment
Projected promotions
Other (please list)
________________________________________________________________________
________________________________________________________________________
SECTION II
Courses Taught
Which of the following continuing education courses does your organization currently teach? For each course taught by your organization, indicate by placing a "" in the appropriate box whether it is taught in a classroom/group setting, through individual correspondence, or in some other manner. The following list includes only the courses required for peace officer continuing education by Section 415.034, Texas Government Code. (Note: Please identify under OTHER, alternative methods of providing training.)
COURSE TAUGHT?
NUMBER COURSE (please circle CLASSROOM CORRESPONDENCE OTHER
one)
3939 Civil Rights, Racial
Sensitivity, and YES NO
Cultural Diversity
3232 Special Investigative
Topics YES NO
including: Recognition
& Documentation of
Cases of Child Abuse,
Child Neglect, Family
Violence, Sexual
Assault, and Issues
Concerning Sex Offender
Characteristics
3737 New Supervisors Course YES NO
3101 Civil Process YES NO
0000 Other Continuing YES NO
Education Courses
SECTION III
Course Evaluation
3939 Civil Rights, Racial Sensitivity, and Cultural Diversity
3232 Special Investigative Topics
3737 New Supervisor's Course
3101 Civil Process
0000 Other Continuing Education Courses
For each of the courses referred to by number in the box above, please indicate with a "" any methods your organization uses to evaluate the course itself.
METHODS OF EVALUATION OF THE COURSE 3939 3232 3737 3101 0000
Impressions and perceptions of academy
administration gained through observation as the
course is being conducted
Impressions and perceptions of the instructors
Impressions and perceptions of the students
Formal critiques prepared by the students at the
completion of training
Cost/benefit analysis
Use of pre-tests and post-tests to evaluate
changes in knowledge, skills and abilities in
the students, based on the training
Use of post-tests only to evaluate changes in
knowledge, skills and abilities in the students,
based on the training
Use of job performance reviews made subsequent
to the training that evaluate whether skills
taught in training are being used on the job by
the students
Other evaluation methods for the course
Please describe:
SECTION IV
Instructor Evaluation
3939 Civil Rights, Racial Sensitivity, and Cultural Diversity
3232 Special Investigative Topics
3737 New Supervisor's Course
3101 Civil Process
0000 Other Continuing Education Courses
For each of the courses referred to by number in the box above, please indicate with a "" any methods your organization uses to evaluate the instructor(s).
METHODS OF EVALUATION OF INSTRUCTOR(S) 3939 3232 3737 3101 0000
Impressions and perceptions of academy
administration gained through observation as the
course is being conducted
Impressions and perceptions of other instructor(s)
Impressions and perceptions of students
Formal critiques prepared by students at the
completion of training
Use of pre-tests and post-tests to evaluate
changes in knowledge, skills and abilities of
students, based on the training
Use of post-tests only to evaluate changes in
knowledge, skills and abilities of students, based
on the training
Use of job performance reviews made subsequent to
the training that evaluate whether skills taught
in training are being used on the job by students
Formal review by management of student-prepared
critiques of the instructor(s), prepared after the
course
Formal review by management of instructor-prepared
lesson plans and/or class examinations to verify
completeness of coverage of learning objectives
Other evaluation methods for instructor(s)
Please describe:
SECTION V
Student Evaluation
3939 Civil Rights, Racial Sensitivity, and Cultural Diversity
3232 Special Investigative Topics
3737 New Supervisor's Course
3101 Civil Process
0000 Other Continuing Education Courses
For each of the courses referred to by number in the box above, please indicate with a "" any methods your organization uses to evaluate the students.
METHODS OF EVALUATION OF STUDENTS 3939 3232 3737 3101 0000
Impressions and perceptions of instructor(s)
gained through observation as the course is being
conducted
Use of pre-tests and post-tests to evaluate
changes in knowledge, skills and abilities of
students, based on the training
Use of post-tests only to evaluate knowledge,
skills and abilities of students, based on the
training
Use of job performance reviews made subsequent to
the training that evaluate whether skills taught
in training are being used on the job by students
Use of skill tests that physically demonstrate
learned abilities
Other evaluation methods for students
Please describe:
SECTION VI
Recommendations
The following methods have been proposed for measuring the quality and effectiveness of continuing education training programs. Which of these measures do you think would be
most effective? (please rate using the following scale)
1 2 3 4
not effective somewhat effective effective very effective
METHODS OF EVALUATION RATING
Impressions and perceptions of academy
administration gained through observation as the
course is being conducted
Impressions and perceptions of the instructors
Impressions and perceptions of the students
Formal critiques prepared by the students at the
completion of training
Cost/benefit analysis
Use of pre-tests and post-tests to evaluate
changes in knowledge, skills and abilities in
the students, based on the training
Use of post-tests only to evaluate changes in
knowledge, skills and abilities in the students,
based on the training
Use of job performance reviews made subsequent
to the training that evaluate whether skills
taught in training are being used on the job by
the students
Other evaluation methods for the course*
Please describe:
* Please attach a copy of any evaluation instruments you have found to be useful.
SECTION VI (continued)
The Commission will include in the report to the legislature, a recommendation for a means of funding the evaluation of continuing education. The following ways have been suggested.
Please indicate with a "" your opinions.
METHOD AGREE DISAGREE
Yearly fees for training licenses or
agreements
Additional Court Costs
(Legislative change to 415.082)
Administrative processing fee based on
course load
If you have other ideas please suggest them below.
Thank you very much for taking the time to complete this questionnaire. We appreciate your comments and suggestions.
Appendix F
TCLEOSE Licensed Academies
Appendix G
TCLEOSE Approved Agreement Trainers
Appendix H
Responding and Non-Responding Academies
ACADEMIES RETURNED
Abilene Police Academy Nueces Co. Sheriff's Academy
Alvin Community College LEA Odessa College Police Academy
Angelina College Police Academy Orange Co. Sheriff's Academy
Arlington Police Academy Panhandle L.E.A.
Austin Police Academy Permian Basin LEA
Baytown Police Academy Pharr Police Academy
Bexar County Sheriff's Academy Plano Police Academy
Capital Area Planning Council Sam Houston State CJ Center
Central Texas Regional Academy San Antonio College LEA
Collin Co. Comm. College LEA San Antonio Police Academy
Dallas Co. Sheriff's Academy San Marcos Police Academy
Dallas Police Academy South Plains Assn of Govt. LEA
Dallas/Ft. Worth Airport Academy South Plains College
Del Mar College Regional Police Academy Southwestern L.E. Institute
East Texas Police Academy Tarleton State University LEA
Eastfield College LEA Tarrant Co. Jr. College Academy
El Paso Co. Sheriff's Academy Temple Police Academy
El Paso Police Academy Texas Alcohol Bev. Comm. LEA
Fort Worth Police Academy Texas A&M University LEA
Galveston Co. Sheriff's Academy Texas City Police Academy
Galveston Police Academy Texas Dept. Public Safety LEA
Garland Police Academy Texas Parks & Wildlife LEA
Grand Prairie Police Academy Texoma Regional Police Acad.
Gus George LEA Tyler Junior College LEA
Hays Co. Sheriff's Academy U.T. Systems Police Academy
Heart of Texas Police Academy UT-Brownsville C. J. Institute
Hildalgo Co. Sheriff's Academy Waco Police Academy
Houston Community College Police Acad. Weatherford College LEA
Houston Police Academy West Central Texas Reg. LEA
Institute of Criminal Justice Studies Wichita Falls Police Academy
Irving Police Academy Northeast Tex. Com. Coll. LEA
Jefferson Co. Sheriff's Academy
Lamar Univ. Police Academy
Laredo Comm. College Regional Academy
Lewisville Police Academy
Longview Police Academy
Lower Rio Grande Valley Academy
Lubbock Police Academy
Middle Rio Grande LEA
Midland Police Academy
Montgomery Co. Sheriff's Academy
Navarro College Police Academy
North Central Texas Reg. Academy
North Harris Montgomery Comm. College
ACADEMIES NOT RETURNED
Alamo Area LEA
Amarillo Police Academy
Brazoria Co. Sheriff's Academy
Brazosport College LEA
Cedar Valley College LEA
College of the Mainland LEA
Concho Valley Academy
Corpus Christi Police Academy
Dallas Baptist Univ. LEA
Del Rio Police Academy
Denton Police Academy
El Paso Comm. Coll. LEA
Galveston Community College LEA
Harris Co. Sheriff's Academy
Killeen Police Academy
Lubbock Co. Sheriff's Academy
McAllen Police Academy
Mesquite Police Academy
Pasadena Police Academy
Richardson Police Academy
San Angelo Police Academy
Sul Ross State University LEA
Tarrant Co. Sheriff's Academy
Travis Co. Sheriff's Academy
University North Texas Police Academy
University of Houston-Downtown LEA
Vernon Regional Jr College LEA
Victoria College LEA
Victoria Police Academy
Wharton Co. Jr. College LEA
Appendix I
Responding and Non-Responding Agreement Trainers
AGREEMENT TRAINERS RETURNED
Addison Police Department Humble Police Department
Alice Police Department Huntsville Police Department
Allen Police Department Hurst Police Department
Austin Airport Police Department Jersey Village Police Department
Austin County Sheriff's Department Johnson County Sheriff's Department
Bay City Police Department Kaufman County Sheriff's Department
Bellaire Police Department La Marque Police Department
Benbrook Police Department Lavaca County Sheriff's Department
Bexar County Constable, Pct. 2 Lake Worth Police Department
Brazos County Constable, Pct. 7 Liberty Police Department
Brazos County Sheriff's Department Llano Police Department
Brenham Police Department Lockhart Police Department
Bryan Police Department McLennan County Sheriff's Department
Burlington Northern Railroad Metro Transit Police Department
Caldwell Police Department North Richland Hills Fire Department
Cameron County Sheriff's Department North Richland Hills Police Department
Classen-Buck Seminars, Inc. Our Lady of the Lake University PD
College Station Police Department Oyster Creek Marshal's Office
Conroe Police Department Paris Police Department
Coppell Police Department Potter County Sheriff's Department
Coryell County Sheriff's Department Rockwall County Sheriff's Department
Crowley Police Department Tarrant Co. District Attorney
Deer Park Police Department TDCJ - Internal Affairs
Denison Police Department Texas A&M University Police Department
Denton County Sheriff's Department Texas Association of Counties
Desoto Police Department Texas Comm. on Jail Standards
El Paso County Constable Pct 2 Texas Municipal Police Association
Euless Police Department Travis Co. District Attorney
Everman Police Department Trophy Club Police Department
Fayette County Sheriff's Department TX/NM College Association
Forest Hill Police Department University of Houston PD
Fort Bend County Constable, Pct. 4 Victoria County Sheriff's Department
Goliad County Sheriff's Department Village Police Department
Gonzales County Sheriff's Department Westworth Village Police Department
Grapevine Police Department Wichita County Sheriff's Department
Gray County Sheriff's Department Young County Sheriff's Department
Harris County Constable, Pct. 2
Harris County Constable, Pct. 3
Harris County Constable, Pct. 8
Highland Park DPS
Hill County Sheriff's Department
Holiday Lakes Police Department
AGREEMENT TRAINERS NOT RETURNED
Angleton Police Department Perryton Police Department
Aransas Pass Police Department Port Lavaca Police Department
Athens Police Department Port of Houston Police Department
Austin Community College Campus P.D. Reeves County Sheriff's Department
Bannon & Associates Rice University Police Department
Bell County Sheriff's Department Robinson Police Department
Beverly Hills Police Department Rockdale Police Department
Bexar County Constable, Pct. 4 Rowlett Police Department
Carrollton Police Department Saginaw Police Department
Cedar Park Police Department Santa Fe Railroad Police Department
Cherokee County Sheriff's Department Seagoville Police Department
Coastal Bend Peace Officer Assn. Sheriff's Association of Texas
Collin County Sheriff's Department Smith County Sheriff's Department
Conroe ISD Police Department South Houston Police Department
Corinth Police Department Southern Methodist Univ. DPS
Dallas ISD Safety & Security Dept. Tarrant County Constable, Pct. 3
Dalworthington Gardens PD Texas District & Co. Attorney Assn.
Eagle Pass Police Department Texas Municipal League
Ellis County Sheriff's Department Texas Police Association
Flower Mound Police Department Texas Tactical Police Officers
Freestone Co. Sheriff's Department The Colony Police Department
Galena Park Police Department Titus County Sheriff's Department
Greenville Police Department TX Crim Just Info Users Group
Harris County Constable, Pct. 1 Union Pacific Railroad PD
Harris County Constable, Pct. 4 Upshur County Sheriff's Department
Harris County Constable, Pct. 6 Ward County Sheriff's Department
Hempstead Police Department Washington County Sheriff's Department
Hillsboro Police Department Webster Police Department
Houston County Sheriff's Department Williamson County Sheriff's Department
Jim Wells Co. Sheriff's Department Wood County Sheriff's Department
Kingsville Police Department
Kleberg County Sheriff's Department
Laredo Police Department
Law Enforcement Television Network
Liberty County Sheriff's Department
Limestone Co. Sheriff's Department
Lower Colorado River Authority
Marshall Police Department
Nacogdoches County Sheriff's Department
Nueces County Const. Pct. 2
Pasadena ISD Police Department
Appendix J
Survey Results
Texas Commission on Law Enforcement
______________________________
QUESTIONNAIRE ON CONTINUING EDUCATION
Overall Response Rate: 154 (61% of 252)
Total Number of All Instructors (including Full-time and Part-time) 21 (average)
Type of Training Provider: 75 Licensed Academy (71% of 105)
79 Agreement Trainer (52% of 149)
Types of law enforcement agencies for which you provide continuing education training
(Please check all that are applicable):
37 (24%) Airport Police Departments
76 (49.4%) College and University Police Departments
89 (57.8%) Constable's Offices
38 (24.7%) County Attorney's Offices
51 (33.1%) District Attorney's Offices
30 (19.5%) Hospital Police Departments
125 (81.2%) Municipal Police Departments
31 (20.1%) Park Police Departments
57 (37%) School District Police Departments
98 (63.6%) Sheriff's Departments
53 (34.4%) State Agencies
34 (22.1%) Other - Please list types:
SECTION I
Training Needs
Every training provider has a method for identifying what type of training is needed for continuing education. Please indicate how you determine this need. (please rank these options using
1 = highest priority)
Averages
3.1 Advisory Board
2.2 Specific departmental (section/division) requests
3.6 Job Analysis
4.6 Instructor availability
4.5 Sources of funding (including fees)
2.0 Legislative requirements
3.4 Other (please list)
Which is most effective?
69 (44.8%) Advisory Board
36 (23.4%) Specific departmental (section/division) requests
10 (6.5%) Job Analysis
2 (1.3%) Instructor availability
1 (.6%) Sources of funding (including fees)
2 (1.3%) Legislative requirements
Once the training needs have been identified, what methods are used to determine if the need has been satisfied?
118 (76.6%) All identified personnel have completed training
31 (20.1%) No demand for training (not signing up for course)
77 (50%) Performance has improved
16 (10.4%) Other (please list)
How do you determine future training needs
113 (73.4%) Regularly Scheduled Needs Assessments
61 (39.6%) Projected new employees
26 (16.9%) Projected new equipment
34 (22.1%) Projected promotions
46 (29.9%) Other (please list)
SECTION II
Courses Taught
Which of the following continuing education courses does your organization currently teach? For each course taught by your organization, indicate by placing a "" in the appropriate box whether it is taught in a classroom/group setting, through individual correspondence, or in some other manner. The following list includes only the courses required for peace officer continuing education by Section 415.034, Texas Government Code. (Note: Please identify under OTHER, alternative methods of providing training.)
COURSE TAUGHT?
NUMBER COURSE (please circle CLASSROOM CORRESPONDENCE OTHER
one)
3939 Civil Rights, Racial YES NO
Sensitivity, and 139 15 133 5 10
Cultural Diversity 90.3% 9.7% 86.4% 3.2% 6.5%
3232 Special Investigative YES NO
Topics 139 15 135 7 11
including: Recognition 90.3% 9.7% 87.7% 4.5% 7.1%
& Documentation of
Cases of Child Abuse,
Child Neglect, Family
Violence, Sexual
Assault, and Issues
Concerning Sex Offender
Characteristics
3737 New Supervisors Course YES NO
78 76 72 4 9
50.6% 49.4% 46.8% 2.6% 5.8%
3101 Civil Process YES NO
63 91 58 2 5
40.9% 59.1% 37.7% 1.3% 3.2%
0000 Other Continuing YES NO
Education Courses 140 14 139 14 18
90.9% 9.1% 90.3% 9.1% 11.7%
SECTION III
Course Evaluation
3939 Civil Rights, Racial Sensitivity, and Cultural Diversity
3232 Special Investigative Topics
3737 New Supervisor's Course
3101 Civil Process
0000 Other Continuing Education Courses
For each of the courses referred to by number in the box above, please indicate with a "" any methods your organization uses to evaluate the course itself.
METHODS OF EVALUATION OF THE COURSE 3939 3232 3737 3101 0000
Impressions and perceptions of academy 87 86 53 36 90
administration gained through observation as 56.5% 55.8% 34.4% 23.4% 58.4%
the course is being conducted
Impressions and perceptions of the instructors 106 102 63 46 104
68.8% 66.2% 40.9% 29.9% 67.5%
Impressions and perceptions of the students 120 115 73 54 120
77.9% 74.7% 47.4% 35.1% 77.9%
Formal critiques prepared by the students at 115 109 77 58 115
the completion of training 74.7% 70.8% 50% 37.7% 74.7%
Cost/benefit analysis 21 22 15 11 30
13.6% 14.3% 9.7% 7.1% 19.5%
Use of pre-tests and post-tests to evaluate 44 38 29 22 50
changes in knowledge, skills and abilities in 28.6% 24.7% 18.8% 14.3% 32.5%
the students, based on the training
Use of post-tests only to evaluate changes in 72 72 46 35 78
knowledge, skills and abilities in the 46.8% 46.8% 29.9% 22.7% 50.6%
students, based on the training
Use of job performance reviews made subsequent 34 33 23 18 45
to the training that evaluate whether skills 22.1% 21.4% 14.9% 11.7% 29.2%
taught in training are being used on the job by
the students
Other evaluation methods for the course 10 10 7 4 10
Please describe: 6.5% 6.5% 4.5% 2.6% 6.5%
SECTION IV
Instructor Evaluation
3939 Civil Rights, Racial Sensitivity, and Cultural Diversity
3232 Special Investigative Topics
3737 New Supervisor's Course
3101 Civil Process
0000 Other Continuing Education Courses
For each of the courses referred to by number in the box above, please indicate with a "" any methods your organization uses to evaluate the instructor(s).
METHODS OF EVALUATION OF INSTRUCTOR(S) 3939 3232 3737 3101 0000
Impressions and perceptions of academy 91 87 58 39 89
administration gained through observation as the 59.1% 56.5% 37.7% 25.3% 57.8%
course is being conducted
Impressions and perceptions of other 69 67 40 33 68
instructor(s) 44.8% 43.5% 26% 21.4% 44.2%
Impressions and perceptions of students 111 109 70 54 114
72.1% 70.8% 45.5% 35.1% 74%
Formal critiques prepared by students at the 118 112 77 59 116
completion of training 76.6% 72.7% 50% 38.3% 75.3%
Use of pre-tests and post-tests to evaluate 34 32 21 18 36
changes in knowledge, skills and abilities of 22.1% 20.8% 13.6% 11.7% 23.4%
students, based on the training
Use of post-tests only to evaluate changes in 63 68 42 33 67
knowledge, skills and abilities of students, 40.9% 44.2% 27.3% 21.4 43.5%
based on the training
Use of job performance reviews made subsequent to 25 24 13 14 35
the training that evaluate whether skills taught 16.2% 15.6% 8.4% 9.1% 22.7%
in training are being used on the job by students
Formal review by management of student-prepared 80 77 54 39 79
critiques of the instructor(s), prepared after 51.9% 50% 35.1% 25.3% 51.3%
the course
Formal review by management of 75 73 52 39 77
instructor-prepared lesson plans and/or class 48.7% 47.4% 33.8% 25.3% 50%
examinations to verify completeness of coverage
of learning objectives
Other evaluation methods for instructor(s) 10 8 7 5 10
Please describe: 6.5% 5.2% 4.5% 3.2% 6.5%
SECTION V
Student Evaluation
3939 Civil Rights, Racial Sensitivity, and Cultural Diversity
3232 Special Investigative Topics
3737 New Supervisor's Course
3101 Civil Process
0000 Other Continuing Education Courses
For each of the courses referred to by number in the box above, please indicate with a "" any methods your organization uses to evaluate the students.
METHODS OF EVALUATION OF STUDENTS 3939 3232 3737 3101 0000
Impressions and perceptions of instructor(s) 106 103 65 51 103
gained through observation as the course is 68.8% 66.9% 42.2% 33.1% 66.9%
being conducted
Use of pre-tests and post-tests to evaluate 42 40 24 23 50
changes in knowledge, skills and abilities of 27.3% 26% 15.6% 14.9% 32.5%
students, based on the training
Use of post-tests only to evaluate knowledge, 81 81 52 41 85
skills and abilities of students, based on the 52.6% 52.6% 33.8% 26.6% 55.2%
training
Use of job performance reviews made subsequent 34 32 23 20 44
to the training that evaluate whether skills 22.1% 20.8% 14.9% 13% 28.6%
taught in training are being used on the job by
students
Use of skill tests that physically demonstrate 45 50 32 30 93
learned abilities 29.2% 32.5% 20.8% 19.5% 60.4%
Other evaluation methods for students 7 6 6 2 11
Please describe: 4.5% 3.9% 3.9% 1.3% 7.1%
SECTION VI
Recommendations
The following methods have been proposed for measuring the quality and effectiveness of continuing education training programs. Which of these measures do you think would be
most effective? (please rate using the following scale)
1 2 3 4
not effective somewhat effective effective very effective
Average
METHODS OF EVALUATION RATING
Impressions and perceptions of academy 2.7
administration gained through observation as the
course is being conducted
Impressions and perceptions of the instructors 2.9
Impressions and perceptions of the students 3.0
Formal critiques prepared by the students at the 3.0
completion of training
Cost/benefit analysis 2.0
Use of pre-tests and post-tests to evaluate 2.9
changes in knowledge, skills and abilities in
the students, based on the training
Use of post-tests only to evaluate changes in 2.6
knowledge, skills and abilities in the students,
based on the training
Use of job performance reviews made subsequent 2.9
to the training that evaluate whether skills
taught in training are being used on the job by
the students
Other evaluation methods for the course* 3.4
Please describe:
* Please attach a copy of any evaluation instruments you have found to be useful.
SECTION VI (continued)
The Commission will include in the report to the legislature, a recommendation for a means of funding the evaluation of continuing education. The following ways have been suggested.
Please indicate with a "" your opinions.
METHOD AGREE DISAGREE
Yearly fees for training licenses or 28 126
agreements 18.2% 81.8%
Additional Court Costs 129 25
(Legislative change to 415.082) 83.8% 16.2%
Administrative processing fee based on 37 117
course load 24% 76%
If you have other ideas please suggest them below.
Thank you very much for taking the time to complete this questionnaire. We appreciate your comments and suggestions.
COMMENTS
FROM QUESTIONNAIRE
ON CONTINUING EDUCATION
Types of law enforcement agencies for which you provide continuing education training.
1. Educators
2. We occasionally host schools in which other agencies participate: Customs, Immigration, Fire Marshals, Sheriff’s Department, Border Patrol, E.M.S., M. P’s
3. Correctional Facilities and Probation Departments
4. Other departments occasionally request to attend our training.
5. Railroad Police
6. Fire Department Arson Investigators
7. LCRA Ranges, Narcotics Enforcement Team, 33rd Dist. S.W.A.T. Team
8. Military Police
9. Marshal offices
10. Have provided training for state, municipal, and federal police officers from the country of Mexico
11. Probation Departments - Adult, Parole Officers, Probation Departments - Juvenile
12. Federal - U.S. Dept. AG
13. Federal agencies (Customs, DEA, FBI, State Dept., INS, Secret Service)
14. Criminal Justice Associate Degree Program (Law Enforcement/Corrections)
15. Bureau of Prisons, Burlington Northern/Santa Fe Police, Community Supervision Officers
16. Federal Agencies
17. U.S. Navy-Base Security (D.O.D), U.S. Park Service, TDCJ - ID
18. Military Police
19. Federal Police
20. Private Security Companies
21. State Correctional System
22. Federal agencies
23. Fire Marshal’s Bailiffs
24. Primarily provide in-house training for Plano Police Department Employees,
25. Special Districts - ie. LCRA, Task Forces
26. Federal, FSIS, International
27. Federal Agencies, Military Police Agencies
28. Only G. P. D. Officers (Galveston PD) no other agencies except Galveston Whalf Police
SECTION I
Training Needs
Every training provider has a method for identifying what type of training is needed for continuing education.
1. Professional judgment as to availability at other locals
2. Legislative requirements have placed an unnecessary burden on our training due to costs to train.
3. Attendance
4. Support, or lack thereof, from County Commissioners Court and it’s recognition of legislative training requirements
5. Requests from officers and supervisors
6. H-GAC bids for Law Enforcement Training Funds
7. Requests of other investigators/any special needs based on impact on investigator safety (this would be possibly # 1 most of the time).
8. Internal Department Training Committee
9. Mandated Training
10. Legislative requirements expend a large portion of the resources and time allocated to training.
11. Per-incident basis: Conduct of members during incident may reveal need for training related to specific incident
12. We only provide training for our department’s employees in those areas covered by our TCLEOSE Training Agreement
13. Analysis of current case law to determine trends in law enforcement court decisions
14. Municipal requirements
15. This agency conducts career counseling with each employee at least one time each year and every employee is given an opportunity give recommendations regarding training they would like to receive and the counselor makes recommendations.
16. Knowledge of needs
17. Through telephone and visitations to area departments/questionnaire
18. Survey of training need inhouse and other agencies
19. Needs not being addressed by other academies in the area (i.e. Reserve Officers)
20. Current social incidents and influences
21. We offer every kind of training possible if our surveys shows a need
22. Employee input (ERC & Nominal Group)
23. Trends in Law Enforcement and Communication
Which is most effective?
1. Professional judgment given our defined role
2. The most effective method is a combination of requests, Job Analysis, Advisory Board, and Legislative Requirements
3. Needs assessment survey with Police Chiefs and Training Coordinators
4. Moot question, Legislative requirements over rides everything anyway.
5. Advisory Board coupled w/an departmental analysis to determine training priorities
6. Specific department (section/division) requests. The individual department heads know better than anyone what type of training is most necessary and effective for the people under their commands.
7. Job Analysis - However there are more and more training mandates placed upon us by state, company and federal - These mandates along with shrinking budgets and increasing time demands - takes away from the available training time.
8. Combination; not one alone is the most effective.
9. Combination of all
10. Specific division and officer requests. The officers and their supervisors generally know better than anyone else what training they need.
11. Knowledge of individual needs
12. Legislative/TCLEOSE mandates
13. Specific departmental request. We do not do job analysis or advisory boards yet. I believe a job analysis would be very effective.
14. Advisory Board/Agency requests/telephone solicitation
15. Each has equal contributing factor
16. The three areas indicated - Advisory Board, Specific departmental (section/division)request, and Legislative requirements - all work together to provide a course of training in which all needs are met.
17. Needs based on trends on and departments request
18. Special departmental requests.
Once the training needs have been identified, what methods are used to determine if the need has been satisfied?
1. Legislative requirements have been fulfilled
2. Whether or not the Academy was able to conduct a satisfactory course
3. Feed back from supervisors, administrators, prosecutors
4. Standardize testing per course
5. If performance
6. The problems have decreased; Less accidents after Pursuit Driving Class
7. Feedback from students and administrators. Does the student utilize training points when dealing w/public, investigating offenses?
8. Feed back from students
9. Observation by supervisors, contact without outside agencies that used our training regarding performance after training
10. Student evaluation & reports from advisory board
11. Frequency of request for course from agencies
12. Evaluation of training
13. Evaluations
14. We do not assess performance of persons after training. Ideally this would be the best way to determine if training is effective
15. Law Enforcement Agencies Response/Feed back
16. Students are provided with a list of course objectives prior to the start of the class. These list specific goals that are to be met during the course of training. Upon completion of the course, students are tested and course evaluations are provided both of which are studied by Academy staff to guarantee training needs are met.
17. Employee Survey and meeting with group
How do you determine future training needs?
1. Feedback from field personnel
2. Input from deputies
3. Formal and informal assessments/surveys is ongoing throughout the department
4. Projected changes in community service needs
5. Verbal & written communication with departments throughout the state.
6. Surveys, polls and our department is currently completing a year long study to determine how we can improve our overall training program throughout the department - from patrol officer to the Chief’s office- including civilians.
7. Projected federal, state and local mandates.
8. Number of other agencies still needing or requesting training - we do needs assessments once or twice a year. We also do a monthly and quarterly report to apprise the department heads what training their people are receiving
9. Projected funds and training needs
10. Departmental requests, Job Analysis, Legislative Requirements
11. Requests from staff, supervisors, officers
12. Visiting with Houston-Galveston area agency heads. Most concerns are related to satisfying State mandates
13. Goals of our town is the main driver in how out officers are trained, along with officer safety.
14. Advisory Board
15. Management/Supervisor Appraisal
16. The requirements mandated by the legislature through TCLEOSE, Advisory Board suggestions, and requests from individual officers
17. Training needs usually based on Law change or technology. Our people are selected from experienced police investigators
18. TCLEOSE requirements and new supervisors/job openings - DARE, etc.
19. Accreditation mandates, policy mandates, TCLEOSE mandates, Training Committee identified
20. Requests from Departmental Supervisors
21. Training mandates
22. Projected transfers and promotions - These identify personnel who may be transferred to specialized units requiring specific training
23., Anticipated change to jail standards and/or legislative changes
24. Needs & recommendations from students, administrators, city managers, courts
25. Plan for mandated training during bi-enniums, train new employees and supervisors as hired. Analyze training trends and curriculum modifications.
26. Job analysis - surveys - mandated training requirements job interest/assignment
27. Local state mandated needs
28. The needs of the fellow officers in what they want for training
29. Reoccurring training needs
30. Legislative requirements and personnel performance
31. Request from department heads and all of the above
32. By keeping advised of current training needs, via legislative requirements and new trends in police training needs.
33. Perceived management needs, examination of training records, continuing review of law enforcement trends and training internal and external
34. Monitor trends around state and country
35. Needs Assessments (not regularly scheduled)
36. TCLEOSE mandates, Training questionnaires from officers
37. Departmental requests
38. By monitoring “Problem” areas on an officers abilities
39. New legislation such as concealed handgun law
40. Fund availability
41. Law Enforcement Agencies requests for specific training courses
42. Problem areas identified through occurrences within department
43. Continous updating information for training from “change”, public to officer requests, and Legislative mandates, Division Requests, Advisory Board
44. Look at trends in policing, current and future issues
45. Formal and informal feedback
46. Division commanders evaluate their needs relaying information to the Academy for future training
47. Employee input thru group survey and advisory Board
48. Different situations that arise - ie. Media coverage on misidentified death in the home prompts a course on death investigation
49. All play a part, however, agencies address legislative training ( when it is realistic). We see ourselves as professional trainers and look toward the trends and low enforcement needs in the community to develop new training.
SECTION II
Courses Taught
1. We rely on instructors in the various fields to teach the respective courses
2. 3737 - We send all new supervisors to a 160 hour police supervision course taught by Southwestern Law Enforcement Institute in addition to a 40 hour In-House school.
3. In the past, we have not taught the 1021 course. We have instead taught the individual components of this course. We have not made plans to teach the 3939 or 3232 courses as of yet, but will probably do so later this year. In the past, we have used LETN for the 1021 course.
It is very difficult for a small department to teach these courses due to the amount of time needed to teach these courses. Scheduling personnel to work while officers are in training is also a major consideration.
4. Officers are sent to other agencies for this training (other)
5. 3232 - We have taught these subjects in the past, however they are inconsistent with RR police assigned duties. We will continue to evaluate the need for instruction - should that change we will continue with 3232.
6. We require new supervisors to attend 3939, 3232 and a First Line Supervisor Course.
7. We are developing courses to be delivered by Distance Learning alternatives. - Multimedia, Video, On Line computer based, TTVN
Problem: TCLEOSE is set up top give credit for number of hours in the classroom - alternative methods require credit be given for objectives. Our Distance Learning program is waiting on TCLEOSE.
SECTION III
Course Evaluation
Methods of Evaluation of the Course
1. Physically demonstrate knowledge and skills learned
2. Annual evaluations are made that address these areas in general but are not specifically tied into the classes.
3. Practical exercise and problems involving application of the new information
4. Feed back from administrators and supervisors
5. Observation of Behavioral change
6. Informal critiques with students at completion of training.
7. Oral review of courses taught. Employee evaluation and general student feedback
8. Student evaluation of course content
9. Course is evaluated by professional evaluator who is on staff at the Police Academy
SECTION IV
Instructor Evaluation
Methods of Evaluation of Instructors
1. Informal meeting with officers
2. Training coordinators sets in on all lessons taught by instructors and critiques after class.
3. In process of developing/incorporating into current performance evaluations
4. Training staff meets to discuss how training sessions went. What worked and what did not.
5. Experience level of instructor
6. Verbal Feedback from personnel
7. Informal critique with students
8. Open discussion during and after training
9. Random observation of classroom lectures by management
10. Classroom monitoring while course is in session
11. Sporadic and un-announced classroom visit by director to evaluate instructions
12. Formal critique of each Instructor at least once a year by training coordinator
13. Instructors are evaluated by professional evaluator who is on staff at the Police Academy
SECTION V
Student Evaluation
Methods of Evaluation of Students
1. Use of pre-presentation discussion to ascertain student knowledge of topic - then compare post test score to measure growth and lesson effectiveness
2. Practical exercises and problems involving application of the new information
3. Supervisor input detailing whether student shows improvement in skill level after training
4. Students personal perception of the course
5. Verbal feedback
6. Skills test are used for some C.E.
7. Role playing, situational summaries
8. Open discussion with students
9. Impressions and perceptions of academy administration gained through observation as the course is being conducted
10. Reduction in FBI Civil Rights Investigations
SECTION VI
Recommendations
Methods of Evaluation
6. Use of pre-tests and post-tests to evaluate changes in knowledge, skills and abilities in the students, based on the training.
(Effective to determine whether course is too remedial for student group)
7. Use of post-tests only to evaluate changes in knowledge, skills and abilities in the students, based on the training
(Difficult to measure what student group already knew going into the course)
8. Use of job performance reviews made subsequent to the training that evaluate whether skills taught in training are being used on the job by the students
(Good but not always possible as many students will not complete)
(Other evaluation methods for the course)
1. Use of verbal check-for-understanding as a pre-test followed by a written post test
2. Suggested form of evaluation:
a. Review of records/complaints specifically related to training areas, i.e.:
(1) Number of complaints filed in I.A. related to cultural indifference, race, - increases decreases since this training began.
(2) Number of reported cases of child abuse, etc. since required training began - clearance rates.
3. Legislative Edict
4. Feed back from administrators and supervisors
5. “Hands on” practical exercises, reviews and performance critiques.
6. Practical skills testing and exercises
7. Impression & perceptions from agency heads as to student performance
8. Practical Application
9. The officers who attended 1021 felt that this is needless training. They enter into the course with a negative attitude and are difficult to instruct
10. Skill Evaluation exercises
11. Oral critiques of class w/suggestions for change
12. Off the record discussions w/students during class breaks; overbearing student comments and attitudes on breaks; follow-up w/agency heads after completion of training
13. Evaluation conducted by professional evaluator
14. Roll call presentation by student over material
*Please attach a copy of any evaluation instruments you have found to be useful
1. Relationship of amount of training provided to Case Clearance rates, I.A. complaints, Community acceptance of officers, officer morale, etc.
2. Send a survey to the special interest groups that caused the legislature to pass mandated training programs. Ask them if the mandate had its intended effect. Ask agency heads if the mandates are having any effects.
The Commission will include in the report to the legislature, a recommendation for means of funding the evaluation of continuing education.
If you have other ideas please suggest them below.
1. Funding the training from State - On number of license peace officers and jailers with each agency
2. Mandated training is to great of a burden on small agencies!
3. Any method that discourages training or penalizes those who complete more training is not acceptable. Yearly fees and administrative fees would definitely be a detriment to municipal academies who fund all of their own training. Regional Academies and Colleges may not be impacted by this since they can pass this on to their consumers.
4. Legislative requirements are imposed on agency heads, however, the political body responsible for providing necessary funds to ensure adequate compliance is not held accountable by the state legislature. Requirements without enforcement are vertually meaningless. No agency head can be held accountable when the funding body refuses to recognize legislative requirements. Law enforcement training and continuing education are absolutely essential to supporting the criminal justice system, yet peace officers and reserve officers in Texas are not disciplined for failing to meet legislative standards. And, political bodies tasked with adequate and proper funding of training are allowed to ignore legislative requirements. Either enforce standards and requirements or do not mandate them. Save the state money by not having an education and standards department!
5. I prefer the Florida method where $1 is kept from every traffic citation paid. The fund pays for all in-service training for police personnel in addition to allowing training providers a chance at submitting grants for the purchase of equipment used for training
6. TCLEOSE has trouble keeping track of what it does now. I have found them to be generally ineffective. I certainly do not believe they should now be keeping track of evaluations. We evaluate our training and use those evaluations to make our training better. TCLEOSE nor the legislature have enough expertise to mandate, designate, or improve the training done by the training academies. I sincerely believe this project is without merit.
7. Repeal Senate Bill 1337
Get the State Legislature out of the Law Enforcement Training business. Let the Governor’s Office, TCLEOSE, and Law Enforcement Professional organizations take care of these matters.
8. The Legislature should get out of the business of mandatory training driven by political means. The Legislature should mandate the number of hours a police officer is trained. Course material should be at the discretion of the Agency’s Chief Administrator, due to the individual needs of the entity in which that agency serves.
9. TCLEOSE should fund and administer State academies and provide trainers to smaller agencies, on request, to standardize training methods and quality of instruction
10. This would just add to the work load to document course load and there is too much paper work already!!
11. Yearly fees for academies only
12. I believe that all mandated courses should be paid for by the state. All instructors should be sent to and course paid for by the state.
Mandated courses have been very costly. Some agencies can not pay for the instructors to attend
13. Yearly fees should be based on size of police agency
14. Legislative grants and finding based on cost/benefit analysis
15. Mandated test for each course. Such as the Basic Peace Officer Course Licensing Examination
16. Also, charge at least $10.00 to take license exam
17. I would suggest that agreement trainers be charged for the “license” to provide training. They make a profit and should pay the fee.
18. Legislation to designate 20% of all seized (narcotic) funds be set aside each year for continuing education of law enforcement officers.
19. Use funds generated from the state lottery, or the upcoming pari-mutuel gambling.
20. Change yearly fee for all peace officer licenses, just like most professional organizations do.
21. More funding needs to be made available to small departments and reserves if training is required
22. If all funds collected for the express purpose of funding law enforcement training were used for that purpose and not diverted to the general fund, we probably wouldn’t have a problem.
More emphasis needs to be put on the quality of training provided by academies and academies held accountable. I think we are looking at their problem from the wrong approach
23. All community colleges receive money from the state. Let these moines be legislated to law enforcement training only
24. As a regional academy located at a college campus, we do not receive any 415.082 funds, and I feel that this is fine. We charge a tuition of $180.00 paid by the police agency.
25. The 415.082 funds were designed for police continuing education and not for police basic courses. I am happy that the legislature has found a way to fund the mandated course requirements.
26. Develop Distance Learning guidelines and a State Driving Program
27. I would like to see state funding for providers of mandated training requirements.
28. We area college academy and the college courses are not considered in the state funded monies now allocated for training. In addition, TCLEOSE needs to do an assessment before allowing just anyone to have an agreement to do training. An example (which I have discussed with Gerald Keown) is that in 1994, Titus County Sheriff’s Office was issued an agreement training license. Our academy is physically in Titus County and only 8 miles from the Sheriff’s Office, but they do not like to send people to us for training because the new Chief of Police is on our Advisory Board, and they do not like him. I personally don’t care about their differences, but I can’t understand why TCLEOSE allowed them to come to have an agreement when we are right here. It makes it appear that TCLEOSE is working against us. Also, if you check their training records, you would see that they have people attended class on the same day, 250 miles apart. It makes me wonder about the validity of their actually having people really sitting there in the training. Recently, their secretary that had been their for 12 years quit because they were making her verify that these people were in training, when in fact they were not. She was worried about lying on government records, so she quit. (Angela Smith) I talked to Angela and asked her why they were so against the college and she said that Ken Schindley, the person the agreement license was issued to, hates the college because we would not hire him to teach. This is true, that he did apply, but because he is not qualified to teach and has been involved in some deals in the past, we would not hire him. At any rate, when I talked with Gerald about this problem, he said that since the Sheriff would not be in office after January 1, 1997, the training agreement would no longer be valid, so this may or may not be worth worrying about. I do wonder how many people really sat in those classes as we have traced 2 work records, and they were working at a job elsewhere when they were supposed to be in class at the Sheriff’s Office. It must be nice! It would certainly help me if I could be two places at once.!
Appendix K
Course Evaluation Instruments
Evaluation for Special Investigative Issues
Evaluation of Child Abuse/Neglect Issues
Please rate your knowledge for each of the topics before and after the class on a scale of one to five.
1 2 3 4 5
No Little Average Above High
Knowledge Knowledge Knowledge Average Knowledge
Knowledge
Course Topic Areas Level of Knowledge Level of Knowledge
Before class After class
Child Abuse
Reporting and
Documenting Child Abuse
Texas Family Code
Comments:
Evaluation of Family Violence Issues
Please rate your knowledge for each of the topics before and after the class on a scale of one to five.
1 2 3 4 5
No Little Average Above High
Knowledge Knowledge Knowledge Average Knowledge
Knowledge
Course Topic Areas Level of Knowledge Level of Knowledge
Before class After class
Spouse Abuse
Family Code
Protective Orders
Reporting and
Documenting Family
Violence
Comments:
Evaluation of Sexual Assault Issues
Please rate your knowledge for each of the topics before and after the class on a scale of one to five.
1 2 3 4 5
No Little Average Above High
Knowledge Knowledge Knowledge Average Knowledge
Knowledge
Course Topic Areas Level of Knowledge Level of Knowledge
Before class After class
Sexual Assault
Victim Reactions
Sexual Assault
Interview Process
Comments:
Evaluation of Issues Concerning Sex Offenders
Please rate your knowledge for each of the topics before and after the class on a scale of one to five.
1 2 3 4 5
No Little Average Above High
Knowledge Knowledge Knowledge Average Knowledge
Knowledge
Course Topic Areas Level of Knowledge Level of Knowledge
Before class After class
Sex Offender
Registration
Sexual Disfunctions
Pedophila
Rape
Sexual Sadistic
Lust Murders
Profiling
Comments:
EVALUATION OF CULTURAL DIVERSITY COURSE
Please rate your knowledge for each of the topics before and after the class on a scale of one to five.
1 2 3 4 5
No Little Average Above High
Knowledge Knowledge Knowledge Average Knowledge
Knowledge
Course Topic Areas Level of Knowledge Level of Knowledge
Before class After class
Relevant Commission Rules
Basic Forms of Prejudice
Your own Cultural Heritage
Workforce Diversity
Human Rights/Discrimination
Communication Styles
Benefits of Valuing Diversity
Effective Cultural Contacts
Do you think you will act or make any decisions differently as a result of this course?
( ) Definitely, many times
( ) Yes, occasionally
( ) Probably
( ) Probably not
( ) No
Comments:
Please rate your knowledge, skills, and abilities for each of the topics before and after the class on a scale of one to five.
1 2 3 4 5
No Little Average Above High
Knowledge Knowledge Knowledge Average Knowledge
Knowledge
Course Topic Areas Level of knowledge, Level of knowledge,
skills and abilities skills and abilities
before class after class
Relevant Commission Rules
Role Identification
Values, Ethics, and Principles
Communication
Leadership Styles
Counseling
Liability and Civil Rights
Issues
Planning and Organizing
Civil rights, Racial
Sensitivity, and Cultural
Diversity
Special Investigative Issues
Issues Concerning Sex Offender
Characteristics
The Transition
Do you think you will act or make any decisions differently as a result of this course?
( ) Definitely, many times
( ) Yes, occasionally
( ) Probably
( ) Probably not
( ) No
Comments: