Sigchi conference proceedings format

Transmitting Narrative: An Interactive Shift-Summarization
Tool for Improving Nurse Communication
Angus Forbes, Mihai Surdeanu, Peter Jansen School of Information, University of Arizona Department of Nursing, University of Arizona ABSTRACT
prevention of adverse health events []. However, unintended This paper describes an ongoing visualization project that consequences mitigate the effectiveness of the EHR [, , aims to improve nurse communication. In particular, we in- ]. Most problematically, nurses alternately taking care of a vestigate the transmission of information that is related to patient are unable to effectively sift through the large amount potentially life-threatening clinical events. Currently these of data available via the EHR in order to find pertinent in- events may remain unnoticed or are misinterpreted by nurses, formation. Studies of nurse behavior has found that many or most unfortunately, are simply not communicated clearly nurses make an effort to talk to each other face-to-face as between nurses during a shift change, leading in some cases they change shifts Ideally, these face-to-face conversa- to catastrophic results. Our visualization system is based on tions provide a way for the responding nurse (leaving the cur- a novel application of machine learning and natural language rent shift) to explain his or her interpretation of the patient’s processing algorithms. Results are presented in the form of an well being to the receiving nurse (starting the next shift). The interactive shift-summarization tool which augments existing absence of this dialog may account for an alarming number Electronic Health Records (EHRs). This tool provides a high of miscommunications that have lead to catastrophic events in level overview of the patient’s health that is generated through patient health [For instance, a report by the Institute of an analysis of heterogeneous data: verbal summarizations de- Medicine [finds that up to 98,000 patients die per year as a scribing the patient’s health provided by the nurse in charge result of complications of therapy due to ineffective commu- of the patient, the various monitored vital signs of the patient, nication and, moreover, that errors in communication cost US and historical information of patients that had unexpected ad- hospitals an estimated $12 billion annually ]. Although verse reactions that were not foreseen by the receiving nurse providing verbal summarizations of patient health during in despite being indicated by the responding nurse. In this pa- the hand-off between responding and receiving nurses can per, we introduce the urgent need for such a tool, describe be helpful, these summarizations can themselves be misin- the various components of our heterogeneous data analysis system, and present proposed enhancements to EHRs via the In this paper, we describe an interactive visualization system shift-summarization tool. This interactive, visual tool clearly that augments EHRs to improve nurse-to-nurse communica- indicates potential clinical events generated by our automated tion. Our system uses a novel application of machine learn- inferencing system; lets a nurse quickly verify the likelihood ing and natural language processing techniques to generate a of these events; provides a mechanism for annotating the gen- series of potential clinical events [and, furthermore, can erated events; and finally, makes it easy for a nurse to navigate offer reasons why these events are plausible based on an anal- the temporal aspects of patient data collected during a shift.
ysis of: the vital signs recorded in the EHR; the verbal “hand- This temporal data can then be used to interactively articu- off” summary of patient health made by the responding nurse; late a narrative that more effectively transmits pertinent data and historical EHR data of patients that had unexpected ad- verse clinical events. In particular we look at data relatedto six categories of clinical events that are most likely to be Author Keywords
a precursor to unexpected patient death: uncontrolled pain, EHRs; text analytics; interactive verification; temporal data; sudden fever, bleeding, changes in respiratory status, changes heterogeneous data alignment; health informatics.
in level of consciousness, and changes in output. We fur-ther describe a novel, interactive shift-summarization visual- ACM Classification Keywords
ization tool that provides: 1) the automatic proposal and noti- H.5.m. Information Interfaces and Presentation (e.g. HCI): fication of these potentially adverse clinical events; 2) a way for nurses to verify the likelihood of these events; 3) a mecha-nism for annotating the automatic proposals and for allowing INTRODUCTION
the nurse to propose their own events; 4) and finally, an inter- The introduction and subsequent adoption of the Electronic active tool that lets the responding nurse associate vital signs Health Record (hereafter, EHR) is a step forward in automat- as they evolve over time with an overview narrative, making it ing data collection useful in the analysis of patient health and easier to indicate highly pertinent data to the receiving nurse.
In effect, our system aims to replicate, to some extent, aspects of face-to-face nurse communication that may have been lost through the introduction of EHRs, and to formalize the ver- bal summarization that some nurses have improvised. That is, our visualization system aims to make it easy for nurses (e.g., [While the SMT analogy is strong, there are clear to create, transmit, and verify narratives about patient health differences between SMT and the task at hand. Most im- that augment their expert knowledge for improved decision portantly, most of the information in the EHR is stored as continuous numeric values (i.e., vital sign measurements),which cannot be directly used in an alignment model. Using Both the development of a computational model to pre- the same SMT analogy, this is equivalent to a language with dict clinical events and the creation of interactive visualiza- an infinite vocabulary (one “word” for each numeric value tion tools based upon this model make use of data gathered of each vital sign). To mitigate this problem, we discretize from real-world scenarios where EHRs are used by nurses the numeric measurements using a three-point scale: below in making decisions regarding patient treatment. Currently, normal, normal, above normal. Because the hand-off reports we have extensively annotated data generated through inter- are considerably more verbose than the EHR, the granularity views with 37 nurses using EHRs who oversaw patients who of the information stored in the two documents varies con- died unexpectedly, and we are streamlining the interview- siderably. For example, an nurse will often choose to ex- ing process in order to gather 100 more samples of nurse plain a single vital sign measurement using one or more sen- (mis)communication ]. Although this work is ongoing, tences. This is different than SMT, where individual words it brings together interdisciplinary expertise in health infor- (or small phrases) in the source language are aligned to sim- matics, machine learning, and text analytics with the aim of ilar words (or phrases) in the destination language. Fortu- creating an effective interactive visualization tool with the po- nately, methods for dealing with this disparity are common in other fields where SMT models are applied to non-translationtasks, such as question answering [where one of the “lan- GENERATING AND EXPLAINING CLINICAL EVENTS
guages” is considerably more verbose. Our alignment model The computational modeling component of our visualization is trained using matched pairs of EHRs and hand-off reports is divided into four interrelated modules, each making use of for a given patient. While SMT models typically require a information available in the EHR and/or the transcriptions of large set of aligned texts in each of the two languages to ar- verbal summarizations, or “hand-off” reports, created by the rive at seamless translations, restricting our SMT model to responding nurse. The first two models provide a detection domain-specific alignments allows us to train using a rela- facility for clinical events, as well as a probabilistic outcome prediction mechanism to determine the relative likelihood ofan undesired outcome given the detection of a clinical event.
Our models produce the following main results (listed below), To make certain that the hand-off report is properly connected which are then displayed or are interactively investigated via with the EHR data, a third model grounds the narrative of the shift-summarization tool. The evaluation criteria for the the hand-off report with vital sign measurements in the EHR.
computational models are a) the success at detecting events, These links between the two content modalities are further and b) the accuracy in predicting patient outcomes. Standard exploited in the fourth model, which generates an automated evaluation of these metrics consists of comparing accuracy (in summary of the hand-off report, highlighting information that terms of sensitivity and specificity) against a gold-standard is predictive of a negative patient outcome.
set of data created by expert human annotators, which we arecurrently in the process of generating. Our inference justi- Previous research, such as [explores the usefulness of fication model will likewise be evaluated against alignments using natural language processing techniques in the context of health informatics and medical decision making. Our systemintroduces the idea of aligning the EHR data that tracks vital health signals from EHR with the verbal hand-off reports cre- Given an electronic health record for a patient, a nurse-to- ated by the nurses. This central feature of our system stems nurse verbal hand-off transcript, or both, the computational from the observation that there are clear connections between model serves as a multiclass classifier able to detect whether the two data sets, and that the responding nurse typically ex- a clinical event has occurred or not. Moreover, the model plains the most relevant aspects of the EHR in the hand-off provides the central predictive features that it used to arrive at report. The information in these two modalities therefore is jointly extracted, such that in the case where a vital sign mea-surement is highlighted in the EHR, the corresponding text in the hand-off report that describes and elaborates on this vital Given an electronic health record for a patient, a nurse-to- sign should also be extracted to provide the receiving nurse nurse verbal hand-off transcript, or both, the computational with more contextually useful diagnostic information.
model generates a probability distribution over a set of possi-ble patient outcomes (neutral, extended length of stay, failure Grounding the text of the hand-off report in the vital sign to rescue). This model also provides the central predictive measurements of the EHR can be viewed as an alignment features used to arrive at this inference.
task that can be formalized as a statistical machine translation(SMT) problem. Conceptually, the EHR and the hand-off re- port can be viewed as as two documents describing the same Given a detection or prediction inference, the computational information in different “languages.” The alignment between model is able to identify specific features in the EHR or hand- these two languages (which grounds the hand-off report in off report that it used to arrive at that inference. Using the terms of the EHR data) can be learned using SMT algorithms grounding of the natural language of the verbal report in EHR data, the model identifies which EHR features align with par- the computational models: the alignment between EHR data ticular sections of the verbal report – providing justification and the responding nurses verbal summarization of the events from multiple sources – or shows where information con- during the shift; a series of generated explanations as to why tained in either the EHR or verbal report was not detected in these events happened; and an overview likelihood, based on the other modality, suggesting a possible error where patient historical data, of whether or not these generated events could information was miscommunicated, incorrectly recorded, or lead to adverse changes in patient condition. Additionally, we incorporate temporal aspects of the EHR data in order to aug-ment the receiving nurses decision making when verifying the EHR and Hand-off Report Summarization recommendations produced by the data analysis module.
Given a hand-off report and corresponding EHR, the compu-tational model generates an automated summary highlighting Specifically, our visualization enhancements provide: a way information that is highly predictive of a negative outcome, to notify nurses about high-risk situations; detailed informa- based on an analysis of historical patient data indicating par- tion that nurses can use to verify these notifications; a way ticular patterns that led to adverse events.
for nurses to meaningfully annotate their interpretation of thepatient health in relation to inferred clinical events; and fi-nally, a system that encourages a nurse to link annotations to Potential for Major Clinical Event
temporal data in order to narrate the overall story of how the patients health evolved over the course of a shift. Each en-hancement builds on the previous, adding functionality that increases the effective transfer of pertinent information be- tween nurses when using EHRs. We are implementing the prototype of our shift summarization tool on an iPad tablet, but expect also to port it to desktop environments or othermobile devices.
Enhancement 1: Shift Summarization (Notify and verify)Our primary enhancement clearly presents the results of ourcomputational analysis. All six of the major clinical events Generated Clinical Events
are listed, and high-likelihood events (as determined via a his-torical analysis of the EHRs of patients who suffered these events) are highlighted. The receiving nurse is thus, at a glance, informed as to whether or not the patient is in im- "weird fluctuation in heart rate" In addition to this high-level overview specific to the major clinical events, we also generate a list of other inferred clin- "weird fluctuation in heart rate" ical events that occurred during the shift and provide the ev- idence as to why they were inferred. That is, for each of the generated clinical events, we link to the EHR monitoring dataor the transcription of the nurse’s verbal summarization that Figure 1. A prototype of the overview page of our shift-summarization led our system to conclude that the event occurred.
tool. At the top we see a simple bar chart indicating high-risk clinicalevents (determined through an evaluation of historical EHR data). At Explicitly presenting data that shows the logic behind the the bottom we see a list of events generated by our system (based on computational inference functions both as a way to create expert rules), alongside related textual snippets from the nurse’s “hand- trust in the system, as has been shown in research on (or im- plemented in) recommendation systems, such as [, ,, ], and also provides a starting point for verifying or VISUALIZATION TOOLS FOR EHRS
invalidating the automatic notifications, which could be espe- The existing design of EHRs is graphically cluttered and pars- cially important in the case of false alarms. A third compo- ing patient data can be cognitively-taxing, even for experts nent of this first enhancement is to provide an interface for [Moreover, in the attempt to provide information about browsing the alignment between EHR data and text. This en- all possible indicators of clinical events, ironically, the most ables the nurses to verify results of our automated system eas- relevant and potentially life-threatening of these can be ob- ily, and also to search freely for patient-specific information fuscated While previous work has explored different that could bolster or invalidate the automated recommenda- approaches to visualizing and managing the complexity of tions. Figure shows a prototype of this enhancement. At EHRs, for example [, our shift-summarization tool pro- the top we see a simple bar chart indicating high-risk clini- vides enhancements to EHRs to improve nurse comprehen- cal events. At the bottom we see a list of events generated sion of and communication about patient data that is related by our system, alongside related textual snippets from the in particular to signals of a sudden and unexpected change in nurse’s hand-off report. Figure shows an example of how a patient condition. The primary components of our visualiza- nurse selecting a particular explanation for a generated clini- tion enhancements are based directly on the data generated by cal event can instantly see more detail about the aligned text and EHR vital signs that lead our system to present this ex- the responding nurse to browse the generated events and an- notations along with the temporal EHR data (e.g. the “flowsheet”) and then to create a visual narrative of how the nurse Enhancement 2: Issue Tracking the EHR (Annotate) reasons about the possibility of clinical events. Narratives are This enhancement aims to promote dialog between the made up of pertinent, sequential events, and providing a sys- nurses, to foster engagement with relevant information about tem that allows the nurse to explain the events of the shift in patient health, and to provide accountability for the nurses a narrative format attempts to replace an important aspect of face-to-face communication that is otherwise lost in the hand- The generated clinical events are essentially an interpreta- off. Figure shows a sketch of the proposed enhancement in tion of the raw data and the nurses verbal summary of the which an annotated timeline is associated with potential clin- shift. Moreover, the alignment process between the EHR ical events. The circles indicate highlighted raw vital signs data and the verbal summary is also based on encoded as- from an EHR flow sheet that led the responding nurse to make sumptions. We provide a system for the nurses to agree or disagree with the automatically generated clinical events and Generated Clinical Event
Textual Summary
Excerpt from Vital Sign Stream
to annotate them with additional pertinent information. The … I noticed that the patient was not sleeping all that annotations are in the form of a predefined comment – such much. We gave the patient more pillows and replaced a flickering light bulb, but the patient still reported some insomnia, with lots of tossing and turning. I also as “agree”, “disagree”, or “inconclusive” – and, optionally, noticed he was irritated and there were some occasions when I observed him muttering to himself. space is available for further detailed commentary. By pro- When he did manage to sleep, he left lots of drool on his pillow. Also, despite an increased dosage of viding a mechanism that operates, essentially, as an issue- fludrocortisone to treat his orthotic hypotension, the patient's pressure seemed unusually low. Also, I continue to monitor the weird fluctuation in heart rate tracker, nurses have an opportunity to create and respond to forums about particular events that may be important to the Figure 2. An example of how a nurse selecting a particular explanation patients health. In issue-tracking software or websites used for a generated clinical event can instantly see more detail about the for software projects (such as ]), this type of commen- aligned text and EHR vital signs that lead our system to present this tary is used to build consensus on interpretation, to expedite decision making, and to facilitate conversation By re-quiring the nurses to annotate each of the generated events Evaluation Methods and Considerations
as well providing the ability to define their own, we instantly Our shift-summarization tool aims to clearly represent the au- create a focused dialog about the relevant issues regarding the tomated interpretation of patient data as well as the sentiment health of the patient. Furthermore, this mechanism creates a of the responding nurse and, second, to ensure that the receiv- trail of accountability, as the nurse can explicitly explain their ing nurses trust the representation sufficiently to incorporate reasons for disregarding an event, or modify the reasoning be- it into their decision making. Our system is meant to augment (and not replace) nurses’ expert skills, and our evaluation isaimed not only at justifying appropriate visualization meth- The enhancement is split into two related components, one ods but also to measure the effectiveness of their integration for the responding nurse and the other for the receiving nurse.
First, we allow the responding nurse to evaluate the shift sum-marization created by Enhancement 1. In particular, we allowthe nurse to annotate the generated events, either with prede- fined terms, or with a more detailed textual explanation. Thusthe receiving nurse has more information regarding consensus between the automated interpretation and the nurses interpre- tation of events in the shift. Second, we similarly provide a space to indicate agreement or disagreement with generated events and allow the receiving nurse to indicate that he orshe has read and the responding nurses annotations as well as space to provide additional commentary.
Enhancement 3: Telling a story via temporal data (Narrate) This enhancement extends the annotation mechanisms de- scribed in Enhancement 2, allowing the responding nurse to link their interpretation of the patient’s health to particular Figure 3. An annotated timeline that helps to tell a story by relating events in time. That is, we allow the responding nurse to pertinent points of data on particular vital sign streams to nurse inter- create a curated timeline of the patient’s health as it evolved over the course of the previous shift. The receiving nursecan then use this temporally-contextualized data to augment Our two main contributions throughout these enhancements his or her decision making process during the current shift.
are a) clearly representing the results generated by our com- Previous research has investigated visual information seeking putational models, and b) providing an interactive interface over temporal data across multiple EHRs , Our with which to support rapid exploration of EHR data. We hy- project emphasizes the temporal aspects of the patient data pothesize that each additional enhancement provides increas- over the course of a single shift. This enhancement allows ingly more effective communication leading to more accurate diagnoses of at-risk patients. We further hypothesize that our 5. Bertram, D., Voida, A., Greenberg, S., and Walker, R.
system increases the nurses ability reason about salient in- Communication, collaboration, and bugs: the social formation, while simultaneously reducing the amount of time nature of issue tracking in small, collocated teams. In they spend filtering out irrelevant data. To test these hypothe- Proceedings of the 2010 ACM conference on Computer ses we plan to run user-studies with nursing students and supported cooperative work, ACM (2010), 291–300.
practicing nurses. These user studies will examine whether an 6. Carrington, J. M. Development of a conceptual expert using the proposed enhancements performs better than framework to guide a program of research exploring the automated system alone. We are also concerned about nurse-to-nurse communication. CIN: Computers, the level of trust an expert might have in the automated sys- Informatics, and Nursing 30, 6 (2012), 293–299.
tem and also with the amount of time it takes to use the en-hancements. Obviously if there is no trust in the system or 7. Carrington, J. M. The usefulness of nursing languages to if it causes an undue burden, then nurses will be less likely communicate a clinical event. CIN: Computers, to use the visualization tools in real-world situations. Thus, Informatics, and Nursing 30, 2 (2012), 82–88.
in addition to the user-studies, we will also conduct cognitive 8. Carrington, J. M., and Tiase, V. L. Nursing informatics walkthroughs in order ascertain the ease of use and potential year in review. Nursing administration quarterly 37, 2 rate of adoption of the tools. As discussed in and [ the iterative design of information visualization tools basedon detailed feedback from cognitive walkthroughs with do- 9. Cebul, R. D., Love, T. E., Jain, A. K., and Hebert, C. J.
main experts can be an effective way to create systems that Electronic health records and quality of diabetes care.
will be useful in real-world environments.
New England Journal of Medicine 365, 9 (2011),825–833.
CONCLUSIONS AND FUTURE WORK
We look forward to a robust discussion regarding a) our ap-
10. Chuang, J., Ramage, D., Manning, C., and Heer, J.
proach to automatically analyzing these interwoven hetero- Interpretation and trust: designing model-driven geneous data and especially our method of aligning vital visualizations for text analysis. In Proceedings of the signs to verbal summaries, and b) our proposed visualiza- SIGCHI Conference on Human Factors in Computing tion enhancements, especially our enhancement that allows a Systems, ACM (2012), 443–452.
nurse to create a narrative out of temporally-evolving patient 11. Cramer, H., Evers, V., Ramlal, S., Van Someren, M., data. While our initial results indicate the effectiveness of Rutledge, L., Stash, N., Aroyo, L., and Wielinga, B. The the relatively straightforward mapping of our automatically effects of transparency on trust in and acceptance of a generated clinical events to visual alerts, we need to more content-based art recommender. User Modeling and thoroughly evaluate whether the additional enhancements are User-Adapted Interaction 18, 5 (2008), 455–496.
worthwhile. Our hope is that the integration of heterogenouspatient data coupled with novel visualization tools will be an 12. Demner-Fushman, D., Chapman, W. W., and McDonald, effective way to transmit a narrative of patient health. Indeed, C. J. What can natural language processing do for whether or not this narrative is perfectly accurate, it should clinical decision support? Journal of biomedical adequately engage with and highlight the most relevant pa- informatics 42, 5 (2009), 760–772.
tient data so that another domain expert (i.e., the receiving 13. Effken, J., and Carrington, J. Communication and the nurse) can quickly validate (or invalidate) it and thereby be- electronic health record: Challenges to achieving the come aware of potential patient risk. Finally, we hope that meaningful use standard. Online Journal of Nursing this additional level of annotation (via the creation of the nar- Informatics (OJNI) 15, 2 (2011).
rative and its validation) itself becomes data that can be takenadvantage of in future analyses that promote accurate inter- 14. Effken, J., and Carrington, J. Strengths and limitations pretation of and communication about patient health.
of the electronic health record for documenting clinicalevents. CIN: Computers, Informatics, and Nursing 29, 6 REFERENCES
1. GitHub. https://github.com, Accessed August 2013.
15. Forbes, A. G., Savage, S., and H¨ollerer, T. Visualizing 2. Google Code. https://code.google.com, Accessed and verifying directed social queries. In Proceedings of IEEE Workshop on Interactive Visual Text Analytics 3. An, J., Wu, Z., Chen, H., Lu, X., and Duan, H. Level of (Seattle, Washington, October 2012).
detail navigation and visualization of electronic health 16. Greenhalgh, T., Stramer, K., Bratan, T., Byrne, E., records. In Biomedical Engineering and Informatics Russell, J., and Potts, H. W. Adoption and non-adoption (BMEI), 2010 3rd International Conference on, vol. 6, of a shared electronic summary record in england: a mixed-method case study. BMJ 340 (2010).
4. Arguello, M., Des, J., Perez, R., Fernandez-Prieto, M., and Paniagua, H. Electronic health records (ehrs) 17. Hyun, S., Johnson, S. B., and Bakken, S. Exploring the standards and the semantic edge: A case study of ability of natural language processing to extract data visualising clinical information from ehrs. In Computer from nursing narratives. Computers Informatics Nursing Modelling and Simulation, 2009. UKSIM’09. 11th International Conference on, IEEE (2009), 485–490.
18. Kohn, L. T., Corrigan, J. M., Donaldson, M. S., et al. To 25. Shneiderman, B. Creativity support tools: Accelerating err is human: building a safer health system, vol. 627.
discovery and innovation. Communications of the ACM 19. Munzner, T. A nested model for visualization design and 26. Sinha, R., and Swearingen, K. The role of transparency validation. IEEE Transactions on Visualization and in recommender systems. In CHI’02 extended abstracts on Human factors in computing systems, ACM (2002), (November-December 2009), 921–928.
20. Needleman, J., Buerhaus, P., Mattke, S., Stewart, M., 27. Stevenson, J. E., Nilsson, G. C., Petersson, G. I., and and Zelevinsky, K. Nurse-staffing levels and the quality Johansson, P. E. Nurses experience of using electronic of care in hospitals. New England Journal of Medicine patient records in everyday practice in acute/inpatient ward settings: A literature review. Health InformaticsJournal 16, 1 (2010), 63–72.
21. Och, F. J., and Ney, H. A systematic comparison of various statistical alignment models. Computational 28. Surdeanu, M., Ciaramita, M., and Zaragoza, H. Learning linguistics 29, 1 (2003), 19–51.
to rank answers to non-factoid questions from webcollections. Computational Linguistics 37, 2 (2011).
22. Plaisant, C., Mushlin, R., Snyder, A., Li, J., Heller, D., and Shneiderman, B. Lifelines: using visualization to 29. Wang, T. D., Plaisant, C., Quinn, A. J., Stanchak, R., enhance navigation and analysis of patient records. In Murphy, S., and Shneiderman, B. Aligning temporal Proceedings of the AMIA Symposium, American data by sentinel events: discovering patterns in Medical Informatics Association (1998), 76.
electronic health records. In Proceedings of the SIGCHIconference on Human factors in computing systems, 23. Pu, P., and Chen, L. Trust-inspiring explanation interfaces for recommender systems. Knowledge-BasedSystems 20, 6 (2007), 542–556.
30. Wang, T. D., Wongsuphasawat, K., Plaisant, C., and 24. Rind, A., Wang, T. D., Aigner, W., Miksch, S., Shneiderman, B. Visual information seeking in multiple Wongsuphasawat, K., Plaisant, C., and Shneiderman, B.
electronic health records: design recommendations and Interactive information visualization to explore and a process model. In Proceedings of the 1st ACM query electronic health records. Foundations and Trends International Health Informatics Symposium, ACM in Human–Computer Interaction 5, 3 (2013).
  • Generating and Explaining Clinical Events

    Source: http://vialab.science.uoit.ca/textvis2013/papers/Forbes-TextVis2013.pdf

    palumbi.stanford.edu

    PERSPECTIVES patterns—electric blue in the westernCaribbean, lemon yellow in the Northern Why Gobies Are Like Hobbits Caribbean—might be local signals to preda-tors that these gobies are the valet serviceAs Bilbo Baggins famously warned, trav- genetic breaks between island groups that rather than the entrée. If this is true, then thelive relatively close together. These breaksw

    bsu.edu.ph

    Benguet State University FUND 911 and Special Projects Statement of Allotment, Obligation & Balances As of December 31, 2012 ALLOTMENT Name of Project A. Special Projects 1. ANEC 2. ATBI (Agri-Based Technology Business Incubator) 3. Bidani 4. Binnadang 5. CHED-HEDF (COD/COE) College of Agriculture 6. CHED - Safe Loan 7. CHED Safe Scholarship 8. Chrysanthemum 9. Cordillera

  • Copyright © 2010-2014 Medical Pdf Finder