|Year : 2018 | Volume
| Issue : 2 | Page : 55-60
Text mining as a method for examining the alignment between educational outcomes and the workforce needs
Jacqueline E McLaughlin1, Carly Lupton-Smith2, Michael D Wolcott3
1 Division of Practice Advancement and Clinical Education, Center for Innovative Pharmacy Education and Research, UNC Eshelman School of Pharmacy, Chapel Hill, NC, USA
2 Whiting School of Engineering, Johns Hopkins University, Baltimore, MD, USA
3 Division of Practice Advancement and Clinical Education, Center for Innovative Pharmacy Education and Research, UNC Eshelman School of Pharmacy; Department of Oral and Craniofacial Health Sciences, UNC School of Dentistry, Chapel Hill, NC, USA
|Date of Web Publication||7-Feb-2019|
Dr. Jacqueline E McLaughlin
Division of Practice Advancement and Clinical Education, Center for Innovative Pharmacy Education and Research, UNC Eshelman School of Pharmacy, Chapel Hill, NC
Source of Support: None, Conflict of Interest: None
Introduction: Job descriptions outline competencies employers consider requisite for success and can be used to inform curriculum design. This paper demonstrates the use of text mining to determine the extent to which key educational outcomes align with job descriptions. Methods: Eighty pharmacist job descriptions containing a total of 12,064 words were text mined using the “bag of words” method. Results: A majority of the 20 most frequent job description words were also present in the pharmacy accreditation standards and American Association of Colleges of Pharmacy entrustable professional activity statements. However, several top 20 words were not, including customer, dispense, department and company. Chi-squared tests revealed text differences between community and health-system pharmacy, including patient, manage, information, and customer, which may have implications for pharmacy schools preparing students for various practice settings. Conclusion: The methodology described here provides a feasible and efficient approach to analyzing current workforce expectations and mapping them to educational outcomes.
Keywords: Competencies, job description analysis, pharmacy, text mining
|How to cite this article:|
McLaughlin JE, Lupton-Smith C, Wolcott MD. Text mining as a method for examining the alignment between educational outcomes and the workforce needs. Educ Health Prof 2018;1:55-60
|How to cite this URL:|
McLaughlin JE, Lupton-Smith C, Wolcott MD. Text mining as a method for examining the alignment between educational outcomes and the workforce needs. Educ Health Prof [serial online] 2018 [cited 2022 Aug 14];1:55-60. Available from: https://www.ehpjournal.com/text.asp?2018/1/2/55/251903
| Introduction|| |
Curriculum redesign is permeating health professions' education amid rapidly changing healthcare practice., Embedded in the challenge of curriculum redesign is understanding the job to be done by practitioners in today's healthcare system. As educators work to align curricula with healthcare practice, they must first determine what knowledge and skills students need to be successful in the workplace.
In pharmacy, educators have applied various research methodologies to identify core competencies for students. These methods have relied largely on published literature, personal experiences, and academic perspectives to determine pharmacy curriculum outcomes.,,,,,
The Academic Affairs Committee of the American Association of Colleges of Pharmacy (AACP) derived entrustable professional activities (EPAs) for pharmacy graduates by brainstorming common activities of pharmacists, consolidating them into a comprehensive list of EPAs, and asking stakeholders to vet them., Similarly, McLaughlin et al. used qualitative research methods to facilitate and analyze discussions between pharmacy preceptors, residents, and faculty to “identify the job to be done by pharmacists as integral members of the healthcare team.”
Job descriptions are another source of data that could be used to determine aspects of pharmacy practice most commonly emphasized by employers. Other disciplines, such as library science and engineering, have used job descriptions to inform curriculum design and student support services.,, Clyde, for example, performed a content analysis of job descriptions to identify librarian competencies and then ensured those skills were being developed through the library curriculum. In addition, Hartmann and Jahren used job posting data to characterize the use of the word “leadership” in an effort to help engineering programs refine curricula and advising materials. This approach has not yet been utilized in health professions' education.
Analyzing job descriptions could provide additional insight into the skills and competencies that healthcare employers consider requisite for job success, which can inform curriculum design and student development. While previous research in pharmacy relied heavily on the perspectives of academics (e.g., pharmacist preceptors and faculty), job descriptions provide direct evidence about the skills current employers consider necessary for pharmacists entering the workforce. As such, the primary aims of the study were to: (1) determine the skills employers are looking for in prospective employees; (2) explore how these skills vary depending on the work setting (i.e., community pharmacy and health-system pharmacy); and (3) examine the extent to which these skills are aligned with key educational outcomes in pharmacy. Specifically, this study targeted pharmacy job descriptions from one state since pharmacist rights can vary from state to state and in an attempt to better understand the job market in which most graduates enter from the researchers' school. This study also serves to introduce readers to text mining as a methodology for health professions' education research and practice.
| Methods|| |
Job descriptions were collected in 2017 (n = 50) and 2018 (n = 30) using the search phrase “Pharmacist in North Carolina” on websites commonly used to advertise pharmacist positions: LinkedIn (n = 64), Indeed (n = 8), Glassdoor (n = 2), CVS Health (n = 2), Simply Hired (n = 1), Delhaize (n = 1), UNC Rex Health (n = 1), and Rx Insider (n = 1). Duplicate job descriptions were removed. The final compilation included 50 unique job descriptions for pharmacists in health-system pharmacy and 30 for community pharmacy. Descriptions were categorized according to work setting due to clear differences in the roles and responsibilities of pharmacists in the two major settings. health systems pharmacists, for example, generally work within hospitals and health clinics and interact more frequently with various healthcare providers to help develop treatment plans, conduct therapeutic drug monitoring (e.g., antibiotics, antidiabetics, and anticoagulants), and to serve as a drug information resource for providers and patients. In contrast, community pharmacists work within retail pharmacies in the community (e.g., Walgreens, CVS) and often focuses on procuring and dispensing medications, offering chronic disease statement management services (e.g., health screening and immunizations), and interacting to a greater extent with patients to educate them as well as refer them to appropriate healthcare providers.
Descriptive statistics were used to examine various characteristics, such as level of position and degree requirements. To prepare the descriptions for the text mining analysis, the requirements, organization/company, location, level of position, and part-/full-time information were removed. All remaining text, which described the job itself, was saved as a “txt” file. The 80 txt files were imported into R Studio (Boston, MA, USA), where they were converted into a corpus file and cleaned using the tm package. Cleaning the text included lowercasing all words and removing punctuation, extra spaces, numbers, and common words such as “and,” “the,” and “a.”
The “bag of words” method of text mining treats each word in a text file as a single entity and does not classify or group words. This method is primarily helpful for comparing word frequencies in descriptive text. Using the “bag of words” method, the corpus containing all 80 job descriptions was converted into a term-document matrix. This term-document matrix contained all words appearing in the job descriptions in the rows of the matrix, while each individual job description made up the columns of the matrix. Using the rowSums function, the total frequency for each word in all of the job descriptions was calculated, and the words were ordered by decreasing frequencies. From the output of words, it was clear that several words needed to be combined or removed so that the final frequencies would be accurate and helpful. For example, the words “drug,” “drugs,” “medication,” and “medications” were combined into “medication.” From this process, the initial mining code was edited to include removing words using the removeWords function and combining words using the gsub function, as needed. With the text organized and cleaned sufficiently, the term-document matrix was made again, and words were ordered by frequency.
To examine the extent to which current job descriptions aligned with key educational outcomes, the most frequently found words were mapped to the Accreditation Council for Pharmacy Education (ACPE) Standards 2016 educational outcomes and the ACCP core EPA statements 2017 collected through the ACPE and AACP websites., Continuous data are presented as mean (standard deviation [SD]), and frequency data are presented as numbers (percent). Chi-square tests were used to examine differences between the words appearing in community pharmacy and health-system job descriptions. Fisher's exact test was used as needed. Statistical significance was set at P < 0.05. IRB approval was not necessary as this study was deemed 'Exempt'.
| Results|| |
Once common words were removed and similar words were combined, the pharmacist job descriptions included a total of 12,064 words, for an average (SD) of 150.8 (188.9) words per job description. The total number of words for health-system pharmacy job descriptions was 5,951 with an average (SD) of 119.0 (91.7) and 6113 words with an average (SD) of 203.8 (279.8) for community pharmacy job descriptions. Most job descriptions were for full-time employment (n = 64, 80%) [Table 1]. There were significant differences between community and health-system job descriptions for the level of position (P = 0.007) and experience (P = 0.0004). Community pharmacy positions were most frequently posted for entry level (46.7%) or not specified (43.3%) while health-system pharmacy positions were most frequently posted for entry level (70%). Community pharmacy positions most frequently did not require experience (60%) while health-system positions most frequently listed preferred experience (34%) or required experience (26%).
|Table 1: Job characteristics of 80 pharmacy practice job descriptions in 2017 and 2018|
Click here to view
There were also differences between community and health-system positions in the frequency and rank order of the most frequent job description words [Table 2]. Chi-square tests showed that health-system positions contained a higher proportion of the words: medication (P < 0.01), patient (P < 0.01), provide (P < 0.01), care (P < 0.01), manage (P < 0.01), usage (P < 0.01), information (P < 0.01), clinic (P < 0.01), and appropriate (P < 0.01) while community positions contained a higher proportion of the words: customer (P < 0.01), ability (P = 0.03), and perform (P < 0.01). The top 5 words in health-system positions were medication, patient, provide, care, and serve while the top 5 words in the community positions were medication, customer, manage, patient, and serve. Therapeutic, system, order, dispense, and technical were in the top 20 for health-system but not in the overall top 20. Department, policy, prompt, and company were in the top 20 for community but not in the overall top 20.
|Table 2: Rank order, frequency, and percentage of all words for the 20 most frequent words|
Click here to view
Since differences were found between the health-system and community pharmacy position descriptions, the top words were mapped to key educational outcomes for all jobs [Table 3], for health-system jobs [Table 4], and for community jobs [Table 5] separately. Most of the top 20 words were present in the ACPE Standards 2016 (n = 13, 65%) and AACP core EPAs 2017 (n = 18, 90%). In the health-system descriptions, most of the top 20 words were present in the ACPE Standards 2016 (n = 14, 70%) and AACP core EPAs 2017 (n = 18, 90%). In the community job descriptions, most of the top 20 words were present in the ACPE Standards 2016 (n = 11, 55%) and AACP core EPAs 2017 (n = 16, 80%). The top words: care and health were present in the educational outcomes. However, the top words: customer, dispense, department, and company were not present in any of the educational outcomes.
|Table 3: The presence of most frequent job description words in key educational outcomes in pharmacy|
Click here to view
|Table 4: The presence of most frequent health-system job description words in key educational outcomes in pharmacy|
Click here to view
|Table 5: The presence of most frequent community job description words in key educational outcomes in pharmacy|
Click here to view
| Discussion|| |
Understanding the knowledge and skills needed for pharmacy practice is critical for developing curricula that prepare students for success in 21st century healthcare. This study demonstrates the utility of a text mining technique to extract information about the current jobs in pharmacy practice. Mapping the results of the text mining to pharmacy education outcomes established by the accrediting body (i.e., ACPE) and prominent pharmacy education organizations (i.e., AACP) provides insight into possible gaps between pharmacy education and pharmacy practice expectations.
Specifically, this study used text mining of job descriptions to determine the skills employers are looking for in prospective employees, how these skills vary depending on the work setting, and the extent to which these skills align with key educational outcomes in pharmacy. The results of the mining procedure revealed that medication, patient, provide, care, and serve were the primary words in health-system positions while medication, customer, manage, patient, and serve were the primary words in the community positions. Furthermore, community pharmacy descriptions involved more usage of customer, ability, and perform while health-system pharmacy jobs related more to medication, patient, and provide. Finally, mapping these skills to notable pharmacy educational outcomes indicated that there was significant agreement between the words found in job descriptions and those found in educational goals for pharmacy students. However, some gaps included the words: customer, dispense, department, and company, which suggests that these terms may lack emphasis in education despite their prominence in the needs of the workforce.
Differences between workplace contexts reflect the growing complexity of pharmacy practice and the challenges schools face as they prepare students for increasingly diverse career pathways. Inconsistencies between community and health-system pharmacy may represent differing norms – patients are believed to be inherently different in terms of their needs and expectations within these two contexts. Community pharmacies, for example, are traditionally more business oriented as it relates to customer service and customers, which could explain why the word “patient” was more frequently used in health-system pharmacist descriptions. These differences may also indicate a lack of consensus regarding competence and skills required in healthcare practice, as found by Teare et al. in a content analysis of nursing job descriptions.
Although most of the job description words were aligned with words from the educational outcome statements, some were notably absent. Most of these missing words were more frequent within community job descriptions, which could indicate that our academy tends to use a vernacular, that is, “patient” centered rather than “customer” centered. It is also possible that curricula are subsequently less focused on pharmacy positions that involve managerial or customer service-oriented responsibilities.
On a broader scope, these missing words lead to important questions about educational practice. If employers use varying terminology, require different skills, and focus on differing requirements, how can schools best identify and accommodate student training needs? Are our outcome statements currently missing important aspects or concepts common to pharmacy practice? To what extent are those possible gaps impacting student preparedness for the job market? Examining why these gaps exist and how they might impact student success are the important next steps for informing the development of pharmacy curricula and improving the alignment between education and practice.
Text mining provides an efficient method for evaluating and informing health professions' curricula. The methodology used in this study could be applied to any profession or text source, including syllabi, rubrics, student feedback, faculty job descriptions, admission materials, patient records, website text, and health professions' education journals. One could text mine course or instructor evaluations, for example, to identify the most common sentiments expressed by students. Course syllabi could be combined within or across schools to determine common elements or topics. Applicant materials (e.g., letters of recommendation, cover letters, and short response questions) from students or residents could be screened to identify themes in the applicants who are selected compared to those who are not to verify alignment with strategic visions of the school or program. Text and keywords from major health professions' journals could also be mined to identify emerging or fading themes, especially in relation to cultural or historical events.
This study uncovered aspects of pharmacy practice considered requisite for employment in today's healthcare system. Our results complement an increasingly important body of research, as practice models rapidly evolve amid healthcare change. Education researchers must continue to play a key role in ensuring that schools understand what, how, and when students need to learn the core competencies necessary to advance healthcare practice. Expanding our methodologies and answering new research questions about educational outcomes could further explicate the needs of students, educators, and schools in the health professions.
Moreover, educators must work with healthcare employers and practitioners to ensure changes in practice and education are paralleled to avoid a mismatch between academic and workforce expectations. Practice change in any setting can stem from various motivators and often presents opportunities to align efforts for progressive change. Research on the origin of clinical practice change, however, rarely identifies education and academia as important components in the process., This raises additional questions concerning what drives change within the profession – workforce needs or academic innovation?
Text mining, specifically, helped elucidate elements and details of practice not previously described in pharmacy education research. Researchers could apply this technique to job descriptions in other health professions, including but not limited to medicine, nursing, dentistry, and veterinary medicine. These studies could employ the “bag of words” approach used in this study or more sophisticated forms of text mining that classify and group words. Future studies could also include additional contexts or a broader scope. Since we excluded job descriptions for pharmacists in industry, regulatory, research, and other alternative career pathways, it is possible that words from those contexts, and their related educational competencies, offer additional insights into student training needs.
When using job descriptions as indicators of contemporary competencies, several potential limitations must be considered. First, the quality of the job descriptions can influence the information gleaned from those descriptions. Employers use various strategies to generate job descriptions, including standardized templates, recycling past descriptions, and writing new descriptions, which can lead to variations in specificity and accuracy. Since job descriptions are external representations of employment needs, employers should accurately emphasize and describe the job to be done so that schools and students can determine which competencies to address and develop. Second, differences in job description norms may exist by workplace setting, company, or position rank, which can influence how competencies are described or understood. Third, interpretation of multiple job descriptions is likely to require combining closely related words, such as medicine and drug. These decisions should be done with caution and vetting, as they could have some influence over how subsequent educational competencies are worded.
This study specifically used the “bag of words” text mining methodology to analyze unstructured text data. A limitation of this approach is the quantitative focus (i.e., number of words), which limits qualitative understanding of the words analyzed. Further, this method focuses on the most frequent words, which may or may not represent the most important words. This may explain why some competencies identified in other types of studies as critical to pharmacy practice, such as communication and adaptability, did not emerge in this study. The quantitative focus of the analysis, however, would enable monitoring job description language over time to explore how pharmacist roles and responsibilities, and their associated skills, change longitudinally. Additional methodologies, including emerging approaches such as machine learning, could also enhance and expand the methodology described here.
| Conclusion|| |
Identifying the competencies needed by graduates can be a complex and time-intensive undertaking. This study demonstrates the use of text mining to examine: (a) what skills and abilities employers are looking for in prospective employees; (b) how these skills and abilities might vary across work settings within a single profession; and (c) the extent to which these skills align with educational outcomes. The results highlight differences in job descriptions between practice settings and between job descriptions and educational outcomes. As schools work to optimize student outcomes, consideration should be given to establishing processes that enable educators to ensure their curricula are contemporary, flexible, and responsive to the rapidly changing needs of the workforce. Text mining provides a relatively simple and efficient method for analyzing large amounts of structured and unstructured workforce data and gleaning useful information for informing educational practice.
Financial support and sponsorship
Conflicts of interest
There are no conflicts of interest.
| References|| |
Roth MT, Mumper RJ, Singleton SF, Lee CR, Rodgers PT, Cox WC, et al.
A renaissance in pharmacy education at the University of North Carolina at Chapel Hill. N C Med J 2014;75:48-52.
Irby DM, Cooke M, O'Brien BC. Calls for reform of medical education by the Carnegie foundation for the advancement of teaching: 1910 and 2010. Acad Med 2010;85:220-7.
Christensen CM, Anthony SD, Berstell G, Nitterhouse D. Finding the right job for your product. MIT Sloan Manag Rev 2007;48:38-47.
Haines ST, Pittenger AL, Stolte SK, Plaza CM, Gleason BL, Kantorovich A, et al.
Core entrustable professional activities for new pharmacy graduates. Am J Pharm Educ 2017;81:S2.
Haines ST, Gleason BL, Kantorovich A, McCollum M, Pittenger AL, Plaza CM, et al.
Report of the 2015-2016 academic affairs standing committee. Am J Pharm Educ 2016;80:S20.
Kennie-Kaulbach N, Farrell B, Ward N, Johnston S, Gubbels A, Eguale T, et al.
Pharmacist provision of primary health care: A modified Delphi validation of pharmacists' competencies. BMC Fam Pract 2012;13:27.
McLaughlin JE, Bush AA, Rodgers PT, Scott MA, Zomorodi M, Pinelli NR, et al.
Exploring the requisite skills and competencies of pharmacists needed for success in an evolving health care environment. Am J Pharm Educ 2017;81:116.
Jungnickel PW, Kelley KW, Hammer DP, Haines ST, Marlowe KF. Addressing competencies for the future in the professional curriculum. Am J Pharm Educ 2009;73:156.
Medina MS, Plaza CM, Stowe CD, Robinson ET, DeLander G, Beck DE, et al.
Center for the advancement of pharmacy education 2013 educational outcomes. Am J Pharm Educ 2013;77:162.
Clyde LA. An instructional role for librarians: An overview and content analysis of job advertisements. Aust Acad Res Lib 2002;33:150-67.
Hartmann B, Jahren C. Leadership: Industry needs for entry-level engineering positions. J STEM Educ 2015;16:13-9.
Maceli M. What technology skills do developers need? A text analysis of job listings in library and information science (LIS) from Jobs.code4lib.org. Inf Technol Libr 2015;34:8-21.
Accreditation Standards and Guidelines for the Professional Program in Pharmacy Leading to the Doctor of Pharmacy Degree: Standards 2016. Accreditation Council for Pharmacy Education; 2015. Available from: https://www.acpe-accredit.org/pdf/Standards2016FINAL.pdf
. [Last accessed on 2018 Nov 07].
Atkinson J, Sánchez Pozo A, Rekkas D, Volmer D, Hirvonen J, Bozic B, et al.
Hospital and community pharmacists' perceptions of which competences are important for their practice. Pharmacy (Basel) 2016;4. pii: E21.
Teare J, Horne M, Clements G, Mohammed MA. A comparison of job descriptions for nurse practitioners working in out-of-hours primary care services: Implications for workforce planning, patients and nursing. J Clin Nurs 2017;26:707-16.
Gupta DM, Boland RJ Jr., Aron DC. The physician's experience of changing clinical practice: A struggle to unlearn. Implement Sci 2017;12:28.
Gregory PA, Teixeira B, Austin Z. What does it take to change practice? Perspectives of pharmacists in Ontario. Can Pharm J (Ott) 2018;151:43-50.
Roberts AS, Benrimoj SI, Chen TF, Williams KA, Hopp TR, Aslani P, et al.
Understanding practice change in community pharmacy: A qualitative study in Australia. Res Social Adm Pharm 2005;1:546-64.
[Table 1], [Table 2], [Table 3], [Table 4], [Table 5]