December 2016
|
December 2016 // Volume 54 // Number 6 // Research In Brief // v54-6rb4
Good Agricultural Practices Certification for Small-Scale Produce Processors: A Case of Food Safety
Abstract
A case study illustrates both the process undertaken by a small produce-processing facility to become certified as having food safety good agricultural practices (GAP) and associated assistance provided by Extension. Information was collected from four U.S. Department of Agriculture Agricultural Marketing Service audits conducted over 2 years. The audits resulted in certification of the facility for processing southern peas and leafy greens at the Harmonized GAP with Global Markets Program Intermediate Level. The case study details the changes the facility implemented to become compliant with the requirements identified during the audits. It was concluded that broad and extensive Extension training and technical assistance could be needed to help small-scale processors become food safety GAP certified.
Introduction
The need for farmers to receive Extension education and training on good agricultural practices (GAP) related to produce food safety continues to increase as marketing opportunities expand and government regulations change. Concurrently, and perhaps consequently, the number of farms that obtain food safety GAP certification through third-party audits also has increased. In many cases, the need for certification is market driven—a situation that occurs when a commercial buyer prefers to or is required to purchase from certified farms to ensure that food is safe, and to avoid lawsuits. This requirement extends along the supply chain, and a buyer purchasing crops from certified farms likely will require that the intermediary postharvest operations (i.e., packers or processors) be similarly certified. This article presents an illustrative case study of the food safety GAP certification of a small-scale processing operation that supported the supply of produce from a group of small farms to a commercial buyer.
Background
The U.S. Department of Agriculture (USDA) Agricultural Marketing Service (AMS) (2015) describes the audit program for produce food safety as follows:
Audit Programs offers [sic] voluntary independent audits of produce suppliers throughout the production and supply chain. Good Agricultural Practices (GAP) and Good Handling Practices (GHP) audits focus on best agricultural practices to verify that fruits and vegetables are produced, packed, handled, and stored in the safest manner possible to minimize risks of microbial food safety hazards . . . [adhering to] U.S. Food and Drug Administration . . . and industry recognized food safety practices. . . . In January 2002, USDA AMS formally implemented the USDA . . . GAP and GHP audit verification program . . . [and] incorporated the Produce GAPs Harmonized Food Safety Standard into its GAP and GHP audit program in 2011. The Produce GAPs Harmonization Initiative is an all-industry effort to harmonize GAP standards.
Regarding this "effort," the United Fresh Produce Association (2015) further clarifies the goal for producers:
The goal . . . is "one audit by any credible third party, acceptable to all buyers . . ." [through] develop[ing] food safety [GAPs] standards and audit checklists for pre- and post-harvest operations, applicable to all fresh produce commodities, . . . on-farm operations and . . . regions . . ., and [making] them available . . . at no cost.
Literature Review
Addressing GAP for produce food safety has been an important area of Extension training and technical assistance for both farmers and farm workers. The standards for certification are high, and, necessarily, Extension activities have been adapted for a variety of audiences so that they are effective and lead to certification. For example, Tobin, Thomson, LaBorde, and Bagdonis (2011) developed a GAP training curriculum based on the results of a survey on the food safety policies of Pennsylvania commercial produce buyers. Their on-farm workshops increased growers' knowledge and confidence and led 20% of participants to plan to seek certification (Nayak, Tobin, Thomson, Radhakrishna, & LaBorde, 2015). Similarly, Kline, Kneen, Barrett, Kleinschmidt, and Doohan (2012) developed a food safety outreach program acceptable for Amish farmers that yielded positive outcomes. Vaughan et al. (2014) assisted over a dozen small-scale and limited-resource farmers in Alabama in becoming certified by undertaking an assortment of activities, including large- and small-group meetings, conference calls, and individual consultations. Mathiasen, Morley, Chapman, and Powell (2012) used a training video to improve agricultural workers' knowledge of food safety and found the approach to be significantly effective. With Iowa growers, Shaw, Strohbehn, Naeve, Domoto, and Wilson (2015) found that a 7-hour GAP course that combined traditional PowerPoint delivery with discussion improved knowledge and attitudes toward food safety.
Methods
The research discussed here was an illustrative case study. Becker et al. (n.d.) defined illustrative case studies as "primarily descriptive" and "typically utiliz[ing] one or two instances of an event to show what a situation is like" ("Types of Case Studies," para. 2). In the research discussed here, the case was a small produce-processing facility, and the event was the process of becoming food safety GAP certified. Information was collected related to audit preparations, the audits themselves, postaudit briefings with the auditors, and the corrective actions needed. The audits were conducted in December 2012, July 2013, December 2013, and July 2014 by the Alabama Department of Agriculture and Industries for USDA AMS. Extension was continuously involved with the process through training and on-site technical assistance, mainly related to translation of the requirements into scale-specific, practicable changes.
Description of the Processing Facility
Facility Specifications
The processing facility, located on a privately owned farm in south-central Alabama, has an area of 2,000 ft2, divided into two equal sections. The front of the building serves as retail space; the rear of the building is the processing area. The processing area has a standard door and a roll-up doorway to a shipping and receiving area under a 5,000-ft2 open shelter. The water and electrical supplies are from the county. The floor in the processing space is unfinished concrete with several floor drains. The processing space has a drop ceiling with smooth, washable tiles. The walls are also washable and have coved bases. There are two bathrooms, one in the retail space and one in the processing area. The processing area has a stainless steel, three-compartment sink and a handwashing sink and is equipped with an ice maker and stainless steel tables. The shipping and receiving area has a 20-by-30-ft cooler, a dock, an equipment storage shed, and a covered storage section for pallets and containers. Figure 1 shows the layout of the facility.
Figure 1.
Layout of Processing Facility
Staffing
The facility is staffed by up to a half dozen workers, mainly Spanish-speaking migrant workers, most of whom understand some English. The food safety training for the workers is supplied in Spanish and English.
Crops Processed
The crops handled at the processing facility include purple hull peas and leafy greens—specifically, collard, kale, mustard, and turnip greens—as well as other vegetables, such as tomatoes, peppers, squash, and onions. Most of the crops are just inspected, packed, stored, and shipped; only a few are actually "processed" (i.e., washed, cut, or shelled) in the facility. Relative to the case discussed herein, the processing facility was GAP-certified for shelled purple hull peas and bundled, crated leafy greens. Inherently, the processing, or work flow, for each of these crops is specific, and the foregoing discussion will be limited to these two GAP-certified crops.
Processing Specifics
Peas
Purple hull peas are a "southern" pea, agronomically and gastronomically similar to black-eyed peas. The hulls, when mature, vary in size from 4 to 10 in. in length and ½ in. in diameter and in color from green with purple mottling to a mostly purple color. Purple hull peas are harvested from June to October.
Figure 2 depicts the work flow for processing peas. The peas arrive at the processing facility whole, in plastic mesh bushel bags. The peas are usually delivered in a refrigerated truck. On receiving, the bags are off-loaded onto pallets and stored in the cooler. A pallet can hold from 25 to 30 bushel bags of unshelled peas.
For processing, the peas are removed from the cooler and loaded into a sheller, which efficiently removes the hull from the edible peas inside. The sheller also removes any immature peas, stems, and other debris. The sheller can shell about 1 bu every 20 min. It can be used continuously for about 4 to 6 hr before it requires cleaning and maintenance. Only two shellers are used in the processing area due to space, personnel, and noise- and air-quality concerns.
The shelled peas are removed from the sheller and run through a blower as well as inspected by hand for the purpose of removing any pieces of hull or debris not removed by the sheller. The inspected, shelled peas are then packed into labeled containers (e.g., clamshells or zippered-closure, clear plastic bags). The label identifies the product, packer, and country of origin and has the item bar code. The containers are then placed by tens into reusable plastic containers (RPCs). RPCs are the ubiquitous black crates found in stores. The RPCs, or crates, are stacked on pallets and stored for shipping in the cooler. The peas are stored in the cooler at 37°F. For shipping, the stacked crates are wrapped with clear stretch wrap, and it is ensured that the vehicle is set to an acceptable temperature.
Figure 2.
Process Flow Diagram for Purple Hull Peas
Note: RPCs are reusable plastic containers.
Leafy Greens
Leafy greens are a southern agricultural staple, and they range among varieties known to grow best in the cooler months. Collard greens, in particular, are a hearty, leafy green with large, smooth leaves and stems. Collard greens can be harvested from October to March.
Figure 3 depicts the work flow for processing collard greens. Collard greens are processed from November through early in the subsequent year. The collard greens arrive, usually by refrigerated truck, already bundled by a rubber band or twist-tie with the buyer's label attached and in RCPs (i.e., crates). The crates are off-loaded at the receiving area and stacked on a pallet. Each crate is first inspected for weight and quality before being topped with 1–2 lb of ice during stacking. The crates are stacked up to five or six levels high and stored in the cooler. Before storage or shipping, the stacks are wrapped with clear stretch wrap.
Figure 3.
Process Flow Diagram for Collard Greens
Results
GAP Certification
Since 2012, the processing facility has been certified for processing southern peas and leafy greens at the USDA AMS Produce Harmonized GAP with Global Markets Program Intermediate Level, which represents the most rigorous audit scheme offered by the USDA. The audit was comprehensive and addressed areas such as worker training, water quality, pest control, and facility maintenance. Becoming certified involved the facility's making numerous changes to typical practices—changes that were determined on the basis of requirements in the audit scheme. Extension rendered assistance to members of the facility's management team by helping them develop a complete and inclusive food safety plan that outlined procedures for making, reviewing, and evaluating the required changes. Required changes and associated Extension assistance pertained to two main areas: (a) personnel and record keeping (Tables 1 and 2) and (b) facilities, equipment, and handling (Tables 3 and 4).
Requirement area | Typical practice(s) before certification | Change(s) needed for certification per the food safety plan |
Management responsibility | Management personnel shared responsibility | Personnel dedicated for food safety management |
Food safety plan or risk assessment | No formal food safety plan established, general cleanliness | Food safety plan developed and reviewed |
Raw material sourcing | Food safety certification not required for supplying farms | Food safety certification required for farms supplying produce to buyers that require certification |
Documentation and record keeping | Record keeping informal and loosely organized | Record keeping on all required aspects of processing operation |
Worker education and training | Training informal and as needed | Training formalized, uniform with industry, and required at hire and annually |
Traceability | Mostly direct-to-consumer sales; some records kept for commercial sales | Developed lot numbers, labeling for containers and pallets, records for produce in and out |
Recall program | Recall possible for most recent sales | Recall records, team, and procedures established and tested |
Corrective actions | Actions taken as needed; general standard | Actions required and recorded; based on food safety standards |
Self-audits | Review of practices as needed | Annual review of all food safety practices |
Requirement area: Change(s) needed for certification | Extension assistance rendered to facility management (time involved) |
Management responsibility: Personnel dedicated for food safety management | Assisted with determining management roles and responsibilities (1 hr) |
Food safety plan or risk assessment: Food safety plan developed and reviewed |
Developed plan template (40 hr) Assisted with completing and reviewing plan (8 hr) |
Raw material sourcing: Food safety certification required for farms supplying produce to buyers that require certification | Assisted with considering policy alternatives and helped develop policy (<1 hr) |
Documentation and record keeping: Record keeping on all required aspects of processing operation | Developed and modified forms and records in template and as result of audit feedback (8 hr) |
Worker education and training: Training formalized, uniform with industry, and required at hire and annually |
Trained managers on food safety (6 hr) Trained employees on food safety (1 hr) |
Traceability: Developed lot numbers, labeling for containers and pallets, records for produce in and out |
Assisted with considering traceability labeling alternatives and with outlining procedures (4 hr) Developed printer label template (<1 hr) |
Recall program: Recall records, team, and procedures established and tested |
Assisted with establishing recall procedures (1 hr) Conducted mock recall exercise (1 hr) |
Corrective actions: Actions required and recorded; based on food safety standards | Assisted with determining actions and timelines for responses to food safety issues and corresponding personnel decisions (<1 hr) |
Self-audits: Annual review of all food safety practices | Assisted with conducting a comprehensive self-audit (4 hr) |
Requirement area | Typical practice(s) before certification | Change(s) needed for certification per the food safety plan |
Agricultural chemicals/plant protection products | Chemicals stored where convenient | All chemicals labeled with dedicated storages areas |
Water/ice | Municipal water used, a tested source | Water tests recorded; facility water system documented |
Containers, bins | Containers stored where convenient | Containers stored to reduce risk of contamination and labeled |
Facility, equipment, tools | Facility, equipment, and tools used with general cleanliness | Facility, equipment, and tools assessed for food safety risk and actions taken and recorded |
Storage | Storage utilized with general cleanliness | Storage assessed for food safety risk and actions taken and recorded |
Waste material | Wastes discarded as needed | Processing wastes and other wastes managed separately |
Outside grounds | Cleaned as needed | Kept cleaned and mowed; trash bins maintained |
Glass control | Lights in building were covered | Lights inside and outside of building and vehicles are checked |
Leaks/lubricants | Problems addressed as needed; no formal plan | Records kept; spill procedures defined |
Equipment and utensil construction | Wood, steel, aluminum, stainless steel, and plastic used | Stainless steel equipment and tables were purchased |
Temporary repairs | Repairs completed as needed by usual standards | Repairs completed and recorded by food safety standards |
Worker health/hygiene and toilet/handwashing facilities | Bathrooms available and cleaned regularly | Records kept of cleaning and stocking of required supplies |
Temperature control | Temperature of cooler checked when entered | Temperature of cooler checked and calibrated and maintenance recorded |
Packing and handling | Packing and handling with general cleanliness | Workers trained on food-safe packaging and handling |
Pest and animal control | Pest control as needed | Professional pest and animal control at least monthly |
Sampling/testing | Water sampling and testing not required, municipal source | Sampling procedures documented in plan |
Packinghouse—water use on produce | Water from municipal source, not tested | Water tests recorded for ice maker source, municipal |
Transportation—temperature control | Refrigerated vehicle typically used, temperature checked | Temperature checked and recorded for shipments |
Transportation—equipment sanitation and maintenance | Vehicles checked for cleanliness | Records kept of vehicle condition |
Requirement area: Change(s) needed for certification | Extension assistance rendered to facility management (time involved) |
Agricultural chemicals/plant protection products: All chemicals labeled with dedicated storages areas | Assisted with identifying suitable storage for chemicals and appropriate labeling (<1 hr) |
Water/ice: Water tests recorded; facility water system documented | Assisted with creating facility water system map (<1 hr) |
Containers, bins: Containers stored to reduce risk of contamination and labeled | Assisted with determining appropriate container storage and handling (<1 hr) |
Facility, equipment, tools: Facility, equipment, and tools assessed for food safety risk and actions taken and recorded | Assisted with inventorying and assessing food safety risks in facility and from equipment and tools (1 hr) |
Storage: Storage assessed for food safety risk and actions taken and recorded | Assisted with assessing food safety risks from storage (<1 hr) |
Waste material: Processing wastes and other wastes managed separately | Assisted with developing wastes management plan for processing and other wastes (1 hr) |
Outside grounds: Kept cleaned and mowed; trash bins maintained | Inspected grounds and identified potential food safety risks (<1 hr) |
Glass control: Lights inside and outside of building and vehicles are checked | Assisted with inventorying potential sources of broken glass/plastic and recommended replacements or procedure changes (<1 hr) |
Leaks/lubricants: Records kept; spill procedures defined | Assisted with identifying risks of leakage and developing spill control procedures (1 hr) |
Equipment and utensil construction: Stainless steel equipment and tables were purchased | Made recommendations for equipment upgrades (<1 hr) |
Temporary repairs: Repairs completed and recorded by food safety standards | Assisted with developing repairs policies, time lines, and actions (<1 hr) |
Worker health/hygiene and toilet/handwashing facilities: Records kept of cleaning and stocking of required supplies | Recommended procedures for cleaning record keeping (<1 hr) |
Temperature control: Temperature of cooler checked and calibrated and maintenance recorded | Recommended procedures and equipment for cooler monitoring (<1 hr) |
Packing and handling: Workers trained on food-safe packaging and handling | Provided training for workers (1 hr) |
Pest and animal control: Professional pest and animal control at least monthly | Assisted with determining pest control needs and documentation requirements (<1 hr) |
Sampling/testing: Sampling procedures documented in plan | Assisted with establishing procedures for sample collection and handling (< 1 hr) |
Packinghouse—water use on produce: Water tests recorded for ice maker source, municipal | Assisted with obtaining appropriate records from county water authority (1 hr) |
Transportation—temperature control: Temperature checked and recorded for shipments | Assisted with developing procedures for ensuring proper vehicle temperatures (<1 hr) |
Transportation—equipment sanitation and maintenance: Records kept of vehicle condition | Assisted with developing procedures for ensuring proper vehicle cleanliness (<1 hr) |
The requirements for the Produce Harmonized GAP with Global Markets Program Intermediate Level certification involved addressing the specific handling of crops, required a hazard analysis critical control points (HACCP) plan (or a written study of work flow addressing the potential for contamination), and required a plan for food defense. Additional changes were needed to meet these supplemental requirements (Table 5). Extension also rendered technical assistance to the management team in making these additional changes (Table 6).
Requirement area | Typical practice(s) before certification | Change(s) needed for certification per the food safety plan |
Food safety plan and documentation | ||
Customer's food safety specifications | Food safety not required outside of general cleanliness | Processing under food safety specification required by customer |
Risk assessment | Risks assessed by general cleanliness | Potential risks assessed according to process and food safety, measures to be taken in plan |
HACCP plan or additional monitoring procedures | General cleanliness and attention to process steps | Additional monitoring procedures in place, similar to HACCP |
Program for nonconforming product | Nonconforming products typically discarded | Actions regarding nonconforming products in plan, recorded |
Corrective actions procedures | Addressed as needed by general standards | Required, recorded by food safety standards |
Nonconformance and complaint record keeping | Addressed as needed by general standards | Actions required and recorded by food safety standards |
Food safety incident procedures | Addressed as needed | Practices in place to address incidents, recorded |
Approved supplier program | Purchases made where convenient | Suppliers recorded and information verified |
Metal detection equipment, if utilized | Not utilized | Not required, may utilize in future |
Food defense | ||
Threat assessment | General access control with signage | Potential threats assessed with food defense plan |
Access control | Primarily signage | Signage, entry and exit recording, enforcement access restrictions |
Sabotage evaluation measures | Determined as needed | Plan to detect sabotage, measures determined when detected, actions recorded |
Note. GAP = good agricultural practices. HACCP = hazard analysis critical control points. |
Requirement area: Change(s) needed for certification | Extension assistance rendered to facility management (time involved) |
Food safety plan and documentation | |
Customer's food safety specifications: Processing under food safety specification required by customer | Assisted with obtaining appropriate documentation from buyer (<1 hr) |
Risk assessment: Potential risks assessed according to process and food safety, measures to be taken in plan | Assisted with identifying potential risks of contamination for each process through dialogue with management (2 hr) |
HACCP plan or additional monitoring procedures: Additional monitoring procedures in place, similar to HACCP | Developed monitoring procedures for all processes and reviewed with management (2 hr) |
Program for nonconforming product: Actions regarding nonconforming products in plan, recorded | Reviewed existing procedures for nonconforming product and recommended changes (<1 hr) |
Corrective actions procedures: Required, recorded by food safety standards | Assisted with determining actions and time lines for responses to food safety issues and corresponding personnel decisions (<1 hr) |
Nonconformance and complaint record keeping: Actions required and recorded by food safety standards | Assisted with determining actions and developing records for nonconformances and complaints (<1 hr) |
Food safety incident procedures: Practices in place to address incidents, recorded | Assisted with determining actions and time lines for responses to food safety incidents (<1 hr) |
Approved supplier program: Suppliers recorded and information verified | Assisted with developing procedures for reviewing suppliers and documenting verification (<1 hr) |
Metal detection equipment, if utilized: Not required, may utilize in future | Assisted with consideration of use of metal detector and options (<1 hr) |
Food defense | |
Threat assessment: Potential threats assessed with food defense plan | Assisted with identifying potential threats (<1 hr) |
Access control: Signage, entry and exit recording, enforcement access restrictions | Assisted with determining placement and wording of signage for access control (<1 hr) |
Sabotage evaluation measures: Plan to detect sabotage, measures determined when detected, actions recorded | Assisted with developing appropriate sabotage evaluation measures and response plan (<1 hr) |
Note. GAP = good agricultural practices. HACCP = hazard analysis critical control points. |
There were, of course, expenses in obtaining food safety GAP certification. The general areas of expense, aside from the audit ($1,000–$2,000), were purchase or rental of new containers, labeling, transportation and storage, equipment upgrades (i.e., stainless steel), handling, water testing, and record keeping and management.
The audits for the facility have been successful, yet there were a few corrective actions needed regarding the water system, container storage, and records of pest control. Otherwise, the operation has been compliant in key areas, such as worker training, water quality, and traceability, which is a concept central to food safety GAP.
Traceability
Because of the importance of traceability to farm of origin, details of compliance in this area are included herein.
For the peas, each pallet of bags received is tagged with a farmer identification sheet, and each RPC shipped is labeled with the crop, the case quantity and container weight, the words "Product of USA," and a lot number that identifies the farm of origin. The lot number is a seven-digit number identifying the date of processing with the three-digit day of the year, the farm with a unique two-digit number, and the pallet with a two-digit number. The label is a commercially available 2-by-4-in. laser or inkjet printer shipping label (Figure 4).
For the leafy greens, each crate received is labeled at the farm with the crop, a lot number that identifies the farm of origin, and the words "Product of USA." The lot number is a seven-character code identifying the date of harvest with the three-digit day of the year, the farm with a unique two-letter code (the farmer's initials), and the field with a two-digit number. The label was a commercially available 2-by-4-in. laser or inkjet printer shipping label (Figure 4).
Figure 4.
Labels Used on Crates for Identification and Traceability to Farm of Origin
Conclusion and Implications
The changes undertaken before and after the audits, with the assistance of Extension personnel, permitted the processing facility to achieve and maintain food safety GAP certification. Extension assistance included food safety materials development, manager and worker trainings, on-site facility evaluation, hours of one-on-one consultations with management, audit observation, and postaudit technical support (Tables 2, 4, and 6).
The main implication of the study is that broad and extensive Extension training and technical assistance may be necessary to assist small-scale processing operations with the process of food safety GAP certification. Extension technical assistance would involve observing and investigating the typical practices at the processing facility, deciphering the numerous and various requirements identified by the auditing organization, and, with the facility management, translating these requirements into needed changes through the development of a food safety plan.
References
Becker, B., Dawson, P., Devine, K., Hannum, C., Hill, S., Leydens, J., . . . Palmquist, M. (n.d.). Case studies. Retrieved from http://writing.colostate.edu/guides/guide.cfm?guideid=60
Kline, T. R., Kneen, H., Barrett, E., Kleinschmidt, A., & Doohan, D. (2012). Adapting Extension food safety programming for vegetable growers to accommodate differences in ethnicity, farming scale, and other individual factors. Journal of Extension, 50(1) Article 1IAW1. Available at: http://www.joe.org/joe/2012february/iw1.php
Mathiasen, L., Morley, K., Chapman, B., & Powell, D. (2012). Using a training video to improve agricultural workers' knowledge of on-farm food safety. Journal of Extension, 50(1) Article 1FEA6. Available at: http://www.joe.org/joe/2012february/a6.php
Nayak, R., Tobin, D., Thomson, J., Radhakrishna, R., & LaBorde, L. (2015). Evaluation of on-farm food safety programming in Pennsylvania: Implications for Extension. Journal of Extension, 53(1) Article 1FEA9. Available at: http://www.joe.org/joe/2015february/a9.php
Shaw, A., Strohbehn, C., Naeve, L., Domoto, P., & Wilson, L. (2015). Knowledge gained from good agricultural practices courses for Iowa growers. Journal of Extension, 53(5) Article 5RIB3. Available at: http://www.joe.org/joe/2015october/rb3.php
Tobin, D., Thomson, J., LaBorde, L., & Bagdonis, J. (2011). Developing GAP training for growers: Perspectives from Pennsylvania supermarkets. Journal of Extension, 49(5) Article 5RIB7. Available at: http://www.joe.org/joe/2011october/rb7.php
United Fresh Produce Association. (2015). GAP harmonization initiative. Retrieved from http://www.unitedfresh.org/food-safety/gap-harmonization-initiative
U.S. Department of Agriculture Agricultural Marketing Service. (2015). Good agricultural practices (GAP) & good handling practices (GHP). Retrieved from http://www.ams.usda.gov/services/auditing/gap-ghp
Vaughan, B., Zeigler, A., Wall, G. D., Robinson, M. D., Hodge, W. A., Bonsi, C. O., . . . & Hill, W. A. (2014). Case study of a food safety/good agricultural practices (GAPs) educational program for small and limited resource produce farmers. Professional Agricultural Workers Journal, 1(2), 1–7. Retrieved from http://tuspubs.tuskegee.edu/pawj/vol1/issue2/7