Ranch hand advisory committee




НазваниеRanch hand advisory committee
страница6/23
Дата17.10.2012
Размер1.21 Mb.
ТипДокументы
1   2   3   4   5   6   7   8   9   ...   23

DR. KAYAJANIAN: Visually.

DR. GOUGH: No, but I mean some of the intervals are    well, like there's an interval 4 to 8, and then there's an interval 8 to 10.

DR. KAYAJANIAN: Yes.

DR. GOUGH: I mean, I can make the intervals 8 to 12, and that would go away, I think.

DR. KAYAJANIAN: It might. Some of the breaks are natural breaks, between this and this, non-detects and detects. The 10 happened to be between, included the background in the comparison group, and then there was this break because Joel had back-calculated the body burden levels to when the men came in as opposed to what they were in '87, which is what describes the first five groups.

DR. GOUGH: So the last point is not even the same kind of data. That's an extrapolated --

DR. KAYAJANIAN: Well, let's put it this way; it may have been extrapolated data, but to be perfectly frank, had I know exactly what the '87 values were, they would have been most probably more appropriate to present. I agree that I could cut the things and these would disappear.

But the point is that I cut them in a way so that you would see a difference. I don't consider that at all dishonest. I mean, if the differences weren't there, you wouldn't see them.

DR. GOUGH: When I draw a line connecting points, which I could do with certainly the first    I could draw any number of lines through those. They're error bars or something, measurement error bars.

DR. KAYAJANIAN: In saying that these are significant, you take the cancer incidence, the number of cancers for however many individuals were in the group; you ask whether that's significantly different from --

DR. GOUGH: And so those error bars don't overlap?

DR. KAYAJANIAN: That's right.

DR. STOTO: Let the record show I'm agreeing with Mike here, having some skepticism on this. Because I don't know about the sample sizes in the Y direction, but I know that the X direction is pretty much random noise below 10.

DR. KAYAJANIAN: If this is random noise, then how do you find this, and as I said--

DR. GOUGH: Well, you make the cuts, Gary.

DR. KAYAJANIAN: Where?

DR. GOUGH: By the way you make    I mean, you   .

DR. HARRISON: The question is being asked, is    let's take from 0 to 10. If you were to make the intervals every two parts per trillion    and sort the incidence by that, would you still see the same shaped curve?

DR. KAYAJANIAN: You probably would blur it. Obviously sine you're doing it in two part intervals, you would certainly see this drop    I'm simply taking your question as --

DR. HARRISON: If you went from 0 to 2, 2 to 4, 4 to 6.

DR. KAYAJANIAN: The zero stands alone.

DR. GOUGH: It's less than 1.25, right?

DR. KAYAJANIAN: Yes, it's less than 1.25. It's sort of a natural break; it's the non-detects.

DR. HARRISON: Okay.

DR. KAYAJANIAN: But if you were to take 1.25 say to 4, then you'd have a point in here. It wouldn't quite be significant, this would be significant. This would be two points, one at 4 to 6, and 6 to 8, and they both    one would be here and the other one would be just a little less than that, and this one would stand.

So what I'm doing is not unreasonable because I had a specific model to test, a predicted, a peak, a very early peak, which turns out to be this, and a later peak; and I expected to find two valleys; one between this and one here.

So it wasn't totally blind and it wasn't totally unfair. The only thing that I could probably be criticized for was splitting this into two. And I would grant you that I split it in order to show that this was a significant drop.

DR. HARRISON: Okay.

DR. KAYAJANIAN: Now the cancers here are, just to be especially clear, they're first cancers, they're first skin cancers plus first non-skin cancers summed. So they're not total, total cancers, but they're the best I could get out of Joel.

DR. MICHALEK: First diagnosis.

DR. KAYAJANIAN: Yes. But if you had a skin and a non-skin, you have both.

DR. MICHALEK: For the record, that last point that you put out there corresponds to roughly 33 parts per trillion, 1987.

DR. KAYAJANIAN: Yes.

DR. MICHALEK: Which is very high relative to the U.S. population.

DR. KAYAJANIAN: Yes, I would have guessed it would have been maybe a bit higher, like 45. I mean, you're putting this break with the middle.

DR. MICHALEK: It then corresponds to the cut between low and high in our reports.

DR. KAYAJANIAN: No, it's higher than that.

DR. MICHALEK: Okay, it's maybe 40.

DR. KAYAJANIAN: And I guess this one would be about 15 or 16.

DR. HARRISON: So what's the take-home lesson?

DR. KAYAJANIAN: The take-home lesson is that whatever conclusions you draw about dioxin at high dose are absolutely irrelevant to people in this background range. Because you cannot extrapolate, either from these first five points as average or from this point, the zero point, to anything out here, because you have significant drops.

So it means, in effect, that all of EPA's high dose observations are worthless, because they're irrelevant. I shouldn't say they're totally worthless, they'd be a value to people out here perhaps, but they're not a value if you're talking about risk assessment policy in this area.

And if you want to reduce dioxin body burdens down to zero from here, you do so by significantly increasing cancer incidence.

DR. HARRISON: The other suggestion that you're making is that you're arguing for a variation in the range from 0 to 10.

DR. KAYAJANIAN: Yes, that's clear.

DR. HARRISON: And Joel has argued that less than 10 gets lumped together.

Am I not right?

DR. MICHALEK: That's what we've done in our reports.

DR. HARRISON: So   

DR. KAYAJANIAN: And in the paper that came out last year where I kept his breaks of 0 to 10 and then splitting the Ranch Handers above 10 into two groups, what you would find if you just took those numbers is that this point is not significantly higher than this, averaging, but this one is significantly lower for non-skin cancers and for melanomas, for which seer reference data are available.

DR. GOUGH: Are these data skin plus?

DR. KAYAJANIAN: Yes, they're skin plus.

DR. GOUGH: That's a very unusual combination. What is it for systemic cancers?

DR. KAYAJANIAN: If you looked at the systemic cancers, you lose some of the --

DR. GOUGH: You lose some numbers.

DR. KAYAJANIAN: You lose some of the significance, but the important thing is, that if you start, for the comparable value here, this is significantly less than this.

DR. HARRISON: Is there a hormonal basis to melanomas? I keep forgetting. It's not a sex-based difference; men greater than women, women greater than men?

DR. KAYAJANIAN: Well, melanomas here constitute about 7-1/2 percent, squamous about 11-1/2. And basal about 82.

That's the first point, and this was in effect, a major portion of the presentation I made to EPA a couple months ago.

The second thing I wanted to talk about was skin cancer, and I want to talk about it not in connection with dioxin; so I'm going to be looking at the zero group, the one that didn't have any dioxin body burden measurements.

There were 117 non-black men, white men in this group, and I looked --

DR. MICHALEK: These are the group with zero parts per trillion?

DR. KAYAJANIAN: Yes.

[Drawing on easel]

Now the number of man years covered in each of these intervals, the first three intervals, is almost exactly the same; and that's quite reasonable, given that in order to be included in the study you had to have a 1987 measurement.

So there were 585 man-years for the first three of these groups. This group, 81 and this group, 15. I'm not really concerned about this, and --

DR. MICHALEK: Why don't you emphasize why you're not so interested in the high dose group.

DR. KAYAJANIAN: Because I want to find out what the effect of being in Vietnam was on skin cancer incidence. So I go where there's no measurable skin cancer.

Now normally if you look at cancer incidence tables as a function of age, you find that as the population ages you get curves that go like this. One form or another; they just go up exponentially, they never go down.

There are some cancers that are more like this, not many. There were no good seer data for basal or squamous cell cancers.

Finding a peak 16 to 20 years after the beginning of the duty tour for each of these men, and these are first skin cancers, is a surprise.

This increase is significant at the .01 level, more importantly this decrease is significant at the .01 level. And you wouldn't find a curve like that unless two things were present. The first thing is that you initiated or you prevented the blocking of initiation, roughly about the time the men began their duty tour. And the second thing you need to have happen is either of two things: Either the promotable skin cancer intermediate is unstable; that is, it doesn't hang around for years and years and years so it can be promoted later; or in the U.S. we have    we're very efficient about promoting cancers. You promote 100 percent of your cancers or close to that at your first opportunity. You wouldn't have a curve like this without it.

I don't know which of those alternative explanations happens to be true, but I think one of them is. And I think this is a very good second way of showing an increase in cancers were associated with service in Vietnam, and not associated with the dioxin.

DR. GOUGH: Is this simply the comparisons?

DR. KAYAJANIAN: This is mostly comparisons. Of the 117 --

DR. MICHALEK: About 90, 80 are comparisons and 24 are Ranch Hands.

DR. KAYAJANIAN: Well    well --

DR. GOUGH: Okay, that's enough Gary. That's what I   .

DR. KAYAJANIAN: I think there were 8 of these, I think 8 of these are from the background group.

DR. MICHALEK: Ranch Hand.

DR. KAYAJANIAN: Yes.

That's the second point, and the third has to do with prostate cancers in black men. And for this [writing new chart] this represents a grouping by dioxin body burden. This is the group that had none, this was the group that had between 1.25 and 4, and this was the group that had more than 4 parts per trillion.

This is the number of black men total, and in these individual groups, this is the number of prostate cancers that were diagnosed. And if you compare this with these together, you have a significant reduction at the .001 level. If you compare this with this, you have a significant reduction at the .1 level; if you compare this with this, you have a significant reduction at the .004 level.

Now let me commence a little bit. When I went through an analysis of this to try to show the dioxin was the cause or at least significantly related to these significant reductions, I made an assumption. And the assumption was that if we took these nine men and assumed that when service began they were 60 years old, so 30 years later, they'd be 90.

So we would expect from each one of these men to have a chance, because they hadn't gone to Southeast Asia; a chance of 6.5, 6.6 percent of having prostate cancer.

So of the three prostate cancers, 9 times 6.6 percent would tell you how many prostate cancers they should have. That hugely upper estimate was .6. If you use that, you have a ratio of 5, and that represents a significant increase.

Joel shared with me, as he came in, more detailed data on when these men were born. So the oldest man would have been 70. So following from 40 to 70 you get roughly .15 cancers expected    and again, that's tilting against the observation I want to make. And in this case, .15 into 3 is 20, so it shows a 20-fold increase in cancers associated with the service in Vietnam or in Southeast Asia.

Even assuming this, which is an extreme worst case, when you make these comparisons, 3 out of 9 as opposed to 1 out of 86, even if this group were much younger than this group, that wouldn't account for very much because all you'd be reducing is that .6 or the comparable number here.

So dioxin is associated with a reduction, and because you have this intermediate point, you show it goes down one.

DR. GOUGH: Gary, did you ever find increases in cancer when you did this analysis with other tumors?

DR. KAYAJANIAN: With   

DR. GOUGH: Any tumors.

DR. KAYAJANIAN: Well, there were increases with    here and here. [chart]

DR. GOUGH: No, no, no. I mean, you've singled out prostate. What about --

DR. KAYAJANIAN: Lung?

DR. GOUGH: Anything. Because I would expect, given these small numbers, you do enough of these analyses and you'll get a pattern like anything.

DR. KAYAJANIAN: Sure, sure. First of all, I haven't looked at a large number of things. With skin cancers you do see the increase. I mean, skin cancer is clearly driving that first set of numbers. So you do see the increase.

And as I said, I expected to see two cancer incidence peaks; one near zero and one near this breakpoint to get you the low, the really low value.

So yes, with skin cancers I clearly saw that. I didn't see it with lung; and if I looked at prostate in white men it would have been pretty flat.

But this is just something that stood out. It's not as if I've done 100 comparisons and I'm bringing you the one that --

DR. GOUGH: Yes, because I think that's what's necessary for analysis like this. These tiny numbers, fluctuating around means--

DR. KAYAJANIAN: But this isn't a fluctuation. Granted it's small, but the point is that you expect something very small here.

DR. GOUGH: I don't, actually, because the Ranch Hands are examined so carefully for prostate cancer, the physical examination    you expect it to be high, as compared to the background population. Particularly black males.

DR. KAYAJANIAN: Well, I understand. I mean, these are the numbers from black males.

DR. HARRISON: So what he's saying is it's an ascertainment bias.

DR. GOUGH: I'm sure there's an ascertainment bias.

DR. HARRISON: That's a function of being involved in the study.

DR. KAYAJANIAN: If that's the case, why don't you see these numbers higher and these numbers higher? I understand. I mean, if you're going to tell me for some reason these men were treated, got much better medical treatment, and so you saw them but you didn't see them here --

DR. GOUGH: No, I'm not going to say that at all. All I'm saying is these are tiny numbers. This .6 and .15 comes from the general population. The seer population.

DR. KAYAJANIAN: No, it comes from the black, seer black, black men in the seer population. And as I said, this was as old as I could ratchet the population up --

DR. GOUGH: Yes, I understood that.

DR. KAYAJANIAN: And this was much more reasonable.

There's also another way of looking at these kinds of data. I didn't know exactly what the age distribution was, say of the comparison group. I couldn't tell you when everyone was born, I couldn't tell you what kind of    how long the length of service was, but I could tell you that for say the comparison group you had the same distribution of age and service in determining skin cancers and determining lung cancers and in determining prostate cancers.

The same population was used, so it's the same distribution. And so what I ended up at one point doing to try to convince Joel that prostate cancers were elevated overall, was to say "Gee, if I look at the ratio of prostate cancer to non-prostate cancers, or to total cancers," total in this case being melanoma plus systemics, and I did it for lung and I did it for say bladder and kidney, and for skin, I'd expect the same ratio as I would find in a certain population grid, or close to it.

The population which described prostate cancer was 75-79, whereas with lung it was more like 40 to 45, and for skin it was more like 35 to 39.

So it's another way of showing that prostate cancer is elevated overall.

So selecting this group wasn't --

DR. GOUGH: Why was the expected, the number I should compare to, the total of seven?

DR. KAYAJANIAN: Well, --

DR. GOUGH: I mean, if I use the upper number, .6 times 86, I'm almost   .

DR. KAYAJANIAN: That's what shows you that it's significantly down.

DR. GOUGH: That doesn't show me anything.

DR. KAYAJANIAN:    with dioxin.

DR. GOUGH: Shows me that you've got exactly the same incidence you'd expect --

DR. KAYAJANIAN: No, but this is --

DR. GOUGH: No, it doesn't. There's a pattern there, but it doesn't    anyway.

DR. KAYAJANIAN: If you take either of these two values, multiply it by 10 --

DR. GOUGH: Come on, Stoto.

DR. KAYAJANIAN:    and multiply it by --

DR. STOTO: None of this makes any sense.

DR. GOUGH: Okay.

DR. KAYAJANIAN:    a ratio of 86:9.

DR. GOUGH: Okay, that's fine.

DR. KAYAJANIAN: If you multiply either of these two numbers by the ratio of 86:9--

DR. STOTO: Can I ask a question? Suppose this were true. I don't understand what the relevance of it is to the Ranch Hand study.

DR. KAYAJANIAN: The relevance I guess is that dioxin reduces --

DR. STOTO: No, No, no. We're trying to design a study and design a set of analyses, and monitor that. Even it were true that dioxin protected against cancer --

DR. KAYAJANIAN: Against this cancer.

DR. STOTO: Even if that were true, what should we do? Should we stop the Ranch Hand study? No, of course not.

DR. KAYAJANIAN: What I would suggest to you to do is instead of using the parameters that you have been using, you rely more on length of service because that's a better reflection    time since the beginning of service, because that's a better reflection of age --

DR. HARRISON: You're saying that after --

DR. KAYAJANIAN:    and so all the talks --

DR. HARRISON:    five years after exposure to dioxin is when all the action starts.

DR. KAYAJANIAN: No. In this case it's not, and I'll tell you why it's not.

DR. HARRISON: Okay.

DR. KAYAJANIAN: You need to have a background of cancers to reduce. This is a small study, and the first 15 years you don't have very much expected cancers, so there's not much to reduce. When you're able to see a reduction in cancers, is when they occur as a result of being in Vietnam.

So for the skin cancers, it's 16 to 20 years you get this peak in the zero group, and then you see a reduction when you move out. And if you want to do it for the non-skin, you look 16 to 30, and you see the reduction.

DR. HARRISON: Okay, so that's --

DR. KAYAJANIAN: What I'm saying is that the parameters you're using are not the best ones.

DR. HARRISON: So that's one thing to consider. We've only got about ten more minutes.

DR. KAYAJANIAN: I'll take any question you want.

DR. HARRISON: No, I just want to make sure that you had finished what you --

DR. KAYAJANIAN: I came to make three points, I made the three points.

DR. HARRISON: Okay. I didn't mean to stifle   

DR. KAYAJANIAN: And I would certainly concede any argument as to the relevance of this, the second one dealing with skin cancers, to what the effect of dioxin is.

I guess the point I want to make here and emphasize is that it's not the dioxin that's doing the damage with respect to cancer; it's having been in Vietnam or Southeast Asia for two to three years. And that's caused roughly a sixfold increase in a lot of cancers. Probably moreso in the case of prostate.

DR. MICHALEK: Two points to make following this is first, if there is an effect of being in Vietnam, we would totally miss it in this study, because we have Vietnam veterans who are controls and we have Vietnam veterans who are an exposed group.

So a pure Vietnam effect would be totally missed. That's one point.

Secondly, the analysis that I proposed earlier, the area under the curve, does accommodate a measure of time since exposure in the model; and it supports the idea that accommodating that body burden of time seems to sharpen our resolve; it's consistent with what he just said.

And thirdly, I have checked the cancer rates, melanoma against the seer rates, and there is roughly a fivefold increase in both groups relative to the seer population. But that analysis itself is open to criticism just like you suggested. But that needs to be argued out as to whether that's recall or not, the so-called ascertainment bias, that we got a very carefully monitored group, and whether that has anything to do with elevating the risk or the ascertainment of cancer. That all needs to be discussed.

That's basically it. And by the way, all the data that he's using is on our web page.

(Laughter)

DR. HARRISON: The nice part about science is that people are going to examine it from every perspective, even unexpected perspectives, and that's their traction of science.

DR. MICHALEK: I always enjoy hearing from Gary.

The other thing to know, for the record, is that all the cancers that he has talked about have been verified two and three hundred percent against medical record review. These cancers are real, they have been diagnosed, and we have documents to prove it.

DR. HARRISON: 300 percent verification. That's --

LTC BURNHAM: That's 100 percent, three times.

(Laughter)

DR. KAYAJANIAN: It's like the vote count in Florida.

DR. STOTO: How many of the cancers were diagnosed at their clinical exams, the Ranch Hands?

DR. MICHALEK: Very few. In fact, because there were no invasive procedures at Scripps for cancer. And I have to check to see the skin cancer; that would be the one that would be the most likely diagnosis at Scripps. We'll have to check that.

DR. HARRISON: So you're saying then, for instance, the prostate cancer data should not then    well, that's not true, because even --

DR. MINER: We're doing PSA.

DR. MICHALEK: You want to know the source of the diagnoses, and we can tell you that.

DR. HARRISON: You have prostate specific antigen; not only that, but the mere fact that every five years these men are contacted and recruited to participate in a health study means that in the interval they are a little more health-aware than the average bear, and if they're not, their doctors damn well are, because they don't want to be embarrassed by having you go out to Scripps and find something that they don't already know about.

DR. STOTO: And that applies to controls as well as the Ranch Hands?

DR. MICHALEK: Yes.

DR. HARRISON: There is still a very reasonable ascertainment issue.

DR. MICHALEK: We call them in between physicals to remind them to go see their doctors. We're on the phone with them all the time, all 2,300 of them. So they get a constant telephone follow-up from us, plus they get letters from Scripps telling them what their abnormalities are, and they are told to take that letter to their family doctor.

So yes, they're watched pretty closely.

DR. HARRISON: Do you have an actual person call them, or do you use one of --

DR. MICHALEK: Our staff.

DR. HARRISON: So you don't use one of these recorded --

DR. MICHALEK: No.

DR. KAYAJANIAN: The ascertainment issue surely shouldn't apply if you're worrying about better medical care. It shouldn't apply across the study; you know, we would apply in comparison with outside groups.

DR. HARRISON: Agreed.

DR. KAYAJANIAN: You don't want to overuse the argument.

DR. MICHALEK: That's true.

DR. HARRISON: Well, thank you very much.

MR. WEIDMAN: I'm Rick Weidman, Vietnam Veterans of America.

DR. HARRISON: Welcome.

MR. WEIDMAN: It's not easy getting in here.

(Laughter)

DR. CAMACHO: Hi, Rick.

MR. WEIDMAN: Hello, Dr. Camacho.

DR. CAMACHO: How are you doing.

MR. WEIDMAN: Good.

DR. HARRISON: Sounds like you're coming out of the pits there every now and then, Paul.

So what do we want to do? It's 10 minutes to 4.

DR. STOTO: Do we have the option of moving forward and doing some of the stuff for tomorrow?

DR. HARRISON: No, not for tomorrow; we're not finished with today. We've got, we did statistical models. We haven't done covariates and interactions, we haven't done analytical planning, general category    I mean, we haven't done any of that stuff.

So do we want to try to press on for maybe another half hour?

DR. CAMACHO: Can I ask a question?

DR. HARRISON: Of course.

DR. CAMACHO: It's about, I was asked to look at 3641. and on, right?

DR. HARRISON: Uh-huh.

DR. CAMACHO: Anyway, you've got a linear model and then it says then that binary variables via logistics regression. I'm just double-checking, you're using the logistic regression to smooth out the variables and get a straight line?

DR. HARRISON: Are you asking Joel?

DR. CAMACHO: I guess so. I'm referring to what    the first document that we were sent, via e mail. And it's 3.6.4.1.1.

DR. HARRISON: It says: These four variables shall be analyzed adjusting for age, race, military occupation?

DR. CAMACHO: Yes, and then it goes to the body fat, the sedimentation rate    I wish somebody would go over that again. Why is that so important? But it's more a case of fat analyzed variables via a general linear model, and then it says: As binary variables via logistic regression.

And the point of their logistics

going to logarithms, is to what, achieve a straight line?

DR. STOTO: No, that's the appropriate method when you have a dichotomous dependent variable. When the Y variable is dichotomous. Either they have the condition or they don't, because you're talking about the risk of getting those   .

DR. CAMACHO: How important is it to have some notion of the true model that's governing the relationship? I mean, we're talking about what we don't know and trying to look at some variables. I don't want to throw a big ringing in the thing because I'm not sure what I'm talking about.

I know that you can pick    what is that reciprocal Ys, you can pick a number of different things. And is it because it's the standard and it makes the data smooth and you get a straight line so you can look at the regression?

DR. STOTO: No, it's not a matter of linearity, it's a matter of the data structure.

DR. HARRISON: So if you're black or non-black, if you're --

DR. CAMACHO: And that's why you're using it?

DR. HARRISON: Yes.

DR. CAMACHO: For that reason.

DR. HARRISON: Yes.

DR. STOTO: You can think about whether you want to put some different curvature on the Xs, but I don't think there's any reason to think that you would want to do that. But when you have a discrete Y with only two values, you pretty much have to use a model like this.

DR. CAMACHO: All right. That'll answer it. I was off on the wrong tangent, then.

DR. HARRISON: So even something like drinking history gets dichotomized?

DR. STOTO: No, this is a dependent variable. Either they have diabetes or they don't or they have heart disease or they don't.

DR. HARRISON: Now since we're on this, this is probably not the appropriate place, but I have no idea why we're still doing sed rates.

DR. MICHALEK: It's not, it's been moved to hematology.

DR. HARRISON: I still don't know why we're doing it at all.

DR. GOUGH: Dr. Camp.

DR. FAVATA: Wanted it? To continue.

DR. GOUGH: Yes, he was very adamant. It was just    over and over and over again he said that it's a general indication of --

DR. HARRISON: I can't argue with anybody that old.

(Laughter)

MR. WEIDMAN: He may be old, but he'll get you   

(Laughter)

DR. HARRISON: Okay. Well, did you have anything else right now, Paul?

DR. CAMACHO: No. Let's progress.

DR. HARRISON: So what about covariates and interactions? That is 3.6.2.

Any comments on that?

DR. STOTO: There are really no changes in that.

DR. HARRISON: What about analytical planning, 3.6.3?

DR. STOTO: I discussed that already; that they're going to use SAS and not have everything on floppies. Perfectly reasonable.

DR. FAVATA: I have some questions and points to raise, Bob. One is, and I apologize for my tardiness and I know that this has been brought up before; but the question is the use of the residential and occupational history regarding chemical exposures post the Vietnam experience.

The issue is that if we only consider the Vietnam experience to be the source of dioxin exposure, and health problems are surfacing right now, we may be inappropriately attributing it to a latent effect; and that may not be the case.

I think that these are very important to include in every analysis. Regarding the residential; Joel, during our October 1999 meeting, and this was from the minutes, page 9 of that meeting, you noted that the EPA has a database of counties in the U.S. which are rank-ordered by a dioxin exposure. And we also have the complete residential history of all the veterans in a very detailed fashion.

So can we therefore access this EPA data and give ourselves the benefit of understanding what their dioxin exposure has been residentially?

DR. MICHALEK: Yes, we can, and we can get the data from the EPA on associating residence with background levels of dioxin exposure. They have that data.

DR. HARRISON: So that was Dr. Favata's residential issue.

DR. MICHALEK: Yes, and that's one of the ideas I forgot.

DR. HARRISON: Not the in-house exposure --

DR. FAVATA: Not within the walls of their house.

DR. HARRISON:    but exposure by zip code or   .

DR. MICHALEK: I thought you were questioning the exposures to chemicals in the household, and you had a series of questions you wanted us to ask them about that. But we can get the EPA data; the answer is yes.

DR. FAVATA: I think that looking at the indoor environment, I really don't think as a group that's going to be a significant exposure to have an effect on this data, and I think that what is most important is from a residential standpoint that we look at their geographic location and see if that has any effect.

DR. STOTO: But given that we have measures of dioxin for these people, is that better than    what's the point of having the residential measures of dioxin.

DR. HARRISON: It might have changed the disappearance rate unexpectedly. You don't have enough points to be sure about that. You might be looking at a 10 that should have gone to a 5 and is still a 10.

DR. FAVATA: And the other thing is that you have an individual with a certain level, you have different rates of elimination; that may be impacted on, we don't know what at this point. So I think, why not utilize that in addition?

Also, regarding their occupational history, in a previous document, and I'm sorry I can't find this    that there is a standard coding system. What is that? What is the standard coding system used?

DR. MICHALEK: For what.

DR. FAVATA: For occupational history.

DR. MICHALEK: There is a    do you know the name of that coding system?

[Simultaneous discussion]

MR. WEIDMAN: There's Standard Industry Code, SIC.

DR. FAVATA: SIC?

MR. WEIDMAN: And the actual occupation is, dictionary of occupational titles was what was formerly used. They just switched over at Department of Labor to a different system, but it's still coded, every occupation in America.

DR. MICHALEK: Exactly. We have a database of all jobs ever held, start and stop dates, in all positions coded according to the system that you just described. We have not made a full effort to relate those occupational exposures to dioxin levels. That's something that should be done eventually.

DR. FAVATA: I think that that would be important.

DR. HARRISON: How are you going to use that, now? How are you suggesting that they use it?

DR. FAVATA: Well, number one, I'm not sure in the SIC codes, whether they    I really need to look at that to see how they break out chemical exposure in those.

DR. STOTO: Not well.

DR. FAVATA: Then that may be a problem if that's --

DR. HARRISON: You'd have to infer it from the occupations. You'd have to --

DR. CAMACHO: A standard book breaks those    I think that you have an industrial category as a group, industrial category and then occupational groups. It breaks down to at least five or six digits.

DR. STOTO: But it could tell you that somebody is in the chemical industry --

DR. CAMACHO: Yes.

DR. STOTO:    but it won't tell you what chemicals were made or whether they work in the factory or in the office or anything like that.

DR. CAMACHO: Yes, I believe so. It gets finer than the usual numbers that we see in a table. I know if you look in the book, it goes down a couple more digits, put it that way. I don't have one in front of me.

DR. HARRISON: But the problem is that you're still only referring to occupation, and the occupation   

DR. CAMACHO: That doesn't necessarily tell you that he worked with it.

DR. STOTO: It's not even occupation, it's the nature of the industry that he worked in and not what the job he had was.

DR. FAVATA: We asked the questions in the neurology section, and we specifically addressed metals, what was in quotes was insecticides and degreasers.

Can't we take that information, since we've already asked it, and then use that as a covariate? Or ask those --

DR. MICHALEK: Yes. In other chapters, right.

MR. WEIDMAN: I guess I'm curious, Doctor, why you particularly would want that, because am I correct in this, that our understanding is that nobody knows what body burden for what period of time causes what toxic, adverse health effects.

Am I correct in that?

DR. FAVATA: You mean in general?

MR. WEIDMAN: In general. Once it's above the threshold.

DR. FAVATA: In dioxin, you're speaking of?

MR. WEIDMAN: Yes, ma'am.

DR. FAVATA: I think that you're correct in saying that.

MR. WEIDMAN: So that if someone lived in Piscataway, New Jersey which has a high dioxin residential rank and worked in the federal chemical agency, it still would not be a counter-indicator as to whether or not the origin to the maladies which that individual suffered, in fact were incurred by the initial exposure in Vietnam.

Am I correct in that?

DR. HARRISON: I think you're coming at it from the wrong angle. We're making the assumption that we're not sure what happens, and we're trying to examine various measures of exposure to dioxin and look at various health measures and see if they correlate in any way; making no presumption that there should be a relationship.

So in essence, we keep asking ourselves, is there anything else we can measure to try and correlate with health effects?

MR. WEIDMAN: Well, I think what you're suggesting is needed.

DR. HARRISON: For example, in Vietnam one of correlations was historical exposure    not in Vietnam, but one of the correlations that's been used in studies is historical exposure. Another one is actual blood dioxin levels, and another one might be whether or not there was continued exposure to dioxin in some form once back Stateside.

Joel has one famous guy who worked on power transformers, and his dioxin levels rose after he returned to the United States.

So you try looking at different exposure parameters to see if any of them provide you with any better insight into whether there's a relationship or not.

MR. WEIDMAN: It's something that clearly we're interested in, too, in the veterans community at large. I had talked with the Iowan, the current committee that just completed their biannual review. And they have urged them to recommend that there be additional studies in the United States of U.S. veterans and not just of Air Force personnel, but a cross-section of people.

And particularly to focus on doing studies of domestic exposure in those counties that you were referring to, Doctor, where we know there's been high exposure of the population in general. And that studies of the population in general in those areas with doing a veteran's identifier, that's never been done.

It hasn't been done at Times Beach or Northeast Arkansas or any of the places that we know were so highly exposed to figure out, is there a cumulative effect? In other words, don't come out and test from this population, but from the broad base of the population with a veteran's identifier or a veteran or progeny of a veteran, an identifier, to see whether or not there's something that leaps generations. Into the next generation or even the grandchildren.

That is within the purview of this committee, as I read 102-4 --

DR. HARRISON: No.

DR. GOUGH: Not at all.

DR. HARRISON: No.

MR. WEIDMAN:    to make recommendations for further research.

DR. HARRISON: Well, it may be within the purview of this committee, but this committee's agenda at this meeting is to review the Statement of Work or scope of work, whichever one it's called, and to make recommendations for, to finalize that scope of work for the Air Force Health Study.

I think you're correct that we're kind of free people, so we can look at this and say that we need other studies. I agree with you wholeheartedly, but within the    if you would like to make some additional proposals, I'll be happy to consider us stopping us right now and reopening the public comment session or, if you're going to be around tomorrow, we can talk about arranging it then so that you get a chance to fully express what you feel we should be considering.

MR. WEIDMAN: That would be very kind sir. I'll come back tomorrow.

DR. HARRISON: Is that acceptable?

DR. GOUGH: Sure.

LTC BURNHAM: It's part of the agenda. On Wednesday.

DR. HARRISON: It is? We've got public again tomorrow?

LTC BURNHAM: Well, no, no. Study disposition.

DR. HARRISON: Oh, okay.

DR. GOUGH: I have a specific question. The dioxin registry that NIOSH has, does NIOSH have comparable registries for people in heavy metals or electroplating or anything like that? Does anyone know?

DR. MICHALEK: There might be ASDR. Do you know?

MR. COENE: I know they have some. They've done a cadmium, they've done --

DR. GOUGH: But they have inventories    the trouble is the numbers, we have so few people, it's probably not worth it.

MR. COENE: They've got some cohort studies in some of the heavy metals.

DR. GOUGH: But they know everybody in the country who is working with dioxin.

DR. HARRISON: What point are we on now and how well are we staying on it?

DR. STOTO: I think the relevance of this is for 3.6.1.5, which has to do with adjustments in the statistical models. When you do an analysis adjusted for, and this is about occupation and body mass index and so on. And then additional things depending on which outcome you're looking at, as described elsewhere.

So I guess one question here should be whether there's any additional information that we want to adjust for of this sort, either everywhere or in some particular analyses. And I guess that I think that what we should be looking at is other things in the occupational environment or the home environment or whatever that we can measure that might be related to the outcome. And that the question about other sources of dioxin is kind of a second order consideration given that we have a measure of dioxin, a body measure. And I think it's worth looking at those things, but I don't think they're the main analysis.

DR. HARRISON: Are you saying that this area-by-area correlation is nice if it can be done, but you don't consider it central? Is that what you're saying?

DR. STOTO: No, no; I think that it makes sense to always adjust for body mass index and to always adjust for occupation, as is suggested here. And then depending on the outcome, what we know about the risk factors for that outcome, we want to adjust for other things. So that all makes sense.

DR. HARRISON: What other things do we want to adjust for?

DR. STOTO: Well, if it's heart disease you want to adjust for    I don't know whether they have exercise or diet or things like that; but those aren't risk factors for other outcomes.

The question is whether there are some general environmental measures other than this military occupation that are worth thinking about or adjusting for always, or even most of the time, or something like that.

DR. HARRISON: The cancer rates are adjusted for obesity, right?

DR. STOTO: Well, they're all adjusted for BMI.

DR. HARRISON: That's all analyses.

DR. STOTO: Yes.

I don't know whether there are good enough data that I'd want to say we should do it on these other things, that we should want to use it all the time.

DR. HARRISON: You know, the reason I thought the area analysis might have made sense is it might affect the persistence of dioxin levels. If that's been taken care of by measuring dioxin levels, then I --

DR. STOTO: I've been pushing generally that we need to think more about how dioxin persists over time, but I'm happy to have that done in these special analyses that the Air Force group does, rather than in the blue book.

Actually, we've been making a distinction here between, the scope of work has to do with the big standardized analysis that the contractor will do, the blue books; and we've recognized that Joel's group can and in fact has been doing other more sophisticated analyses that they publish in journals.

So I think that your idea is a good one to look at, but I'm not sure there are ideas for the blue book.

DR. HARRISON: And one reason would be that you'd either want to go back and do all the prior health study years so that there was some longitudinal comparison, or you'd have a stand-alone analysis, which is really the kind of thing that Joel and his group, and his collaborators have been publishing as they got to it.

I guess that's the --

DR. STOTO: That's part of it, yes.

DR. HARRISON: Yes.

DR. FAVATA: Well, if we have that information anyway, we ask those questions regarding occupation. Then    I think that it's foolish not to use it in this analysis.

I don't know, how    what are the logistics of   ?

DR. MICHALEK: I'm not rebelling against your point, and I agree. I think it's certainly within    it's feasible to do that in a blue book. Because remember in the big blue book we're doing everything unadjusted and then adjusted. So we always have a track record compared to previous reports on the unadjusted analyses.

And based on, let's face it, 15 years' worth of analysis, we've never seen any wild changes after adjustment. You know, like the picture suddenly became clear and everything was supposed to clear    Wow, you know, like we're really on the track because we put a covariate in the model. That has never happened.

DR. GOUGH: I agree. I agree with Elissa, although I don't generally favor adding on analyses. We collected these data.

DR. MICHALEK: Why not use them.

DR. GOUGH: Why not use them. And particularly they bear    a common confounder.

DR. HARRISON: Let's just think about this for a moment, though. Do you want to just use where the person's living now?

DR. GOUGH: No, no. I'm talking about the interview. I'm talking about the interview that's in the neurology session about, "have you ever worked with?"

DR. HARRISON: Oh.

DR. GOUGH: That's what I was talking about.

DR. HARRISON: Oh, I'm sorry. I'm a day late and a dollar short.

DR. STOTO: We've actually been talking about a number of different thing   

(Laughter)

-- but I like that proposal as well.

DR. HARRISON: Okay, in the neurology interview. Okay, all right. That's a separate question.

DR. GOUGH: Really? You're losing your grip, we were talking about more than one thing, Bob.

DR. STOTO: We were talking about those covariates, but using them across the board, right?

DR. HARRISON: The thing that I was concerned about was the area analysis, because I could see as you thought about it, you'd want an average time that a person spent in each area. You know, you'd have to sort that out as well as not just at a single point in time; and Joel was talking earlier about in the matching analyses that's a complicated programming kind of question that would not be easily done by--.

DR. MICHALEK: Let's follow this thought through for a second more about the residential location versus the EPA databases on environmental levels.

DR. FAVATA: Bob does raise a good point as far as the duration of time that they served.

DR. MICHALEK: We have the entire residential history in our dataset; the start and stop date of every residence by longitude, latitude and zip code. It's all there, right back to birth, I think.

Now if I went to EPA and said "Well, I'm going to accumulate, county-by-county, by taking the time spent in a given county times the dose for that county and adding them all up, they would cringe." And the reason they'll cringe is that number one, their dataset is pieced together from many different small studies, from many different sources, from soils, from animals, from whatever; and they would see that as way, perhaps, an overextension of the intended purpose of their analysis.

And then they would also object that "Well, that may be what the U.S. looked like in 1992, it doesn't mean that's what it looked like in 1982." So they would be very, probably very uneasy with this kind of approach.

I'm talking about Linda Birnbaum, Bruce Rhoded and Mike DeVito at EPA.

DR. STOTO: But it's a very different argument for the kind of other exposures that you collect in your questionnaire.

DR. GOUGH: Occupation.

DR. MICHALEK: Yes, let's talk about that for a second, too. It's very easy for us to get a little bit too overconfident about these covariates. When we say, "Have you ever been exposed to heavy metals? Yes or No." A lot of guys say what? What's a heavy metal?

The NORC interviewer, she can't elaborate. She's not a chemist, she's not an occupational-environmental engineer, she just reads the question. And he'll answer yes or no. "Were you ever exposed to degreasing chemicals?" Well, a guy who works in a factory who has worked with cars or equipment might know what she's talking about, but otherwise they don't know what you're talking about.

And so you've got a simple binary response, yes or no, to a question that may not have been understood in the first place. This is very soft data. Anyone who works in the field knows this. However, this is the only study in the world that has this data. So you might say "Okay, well, we've got it, let's use it." Well, we do use it. We use it in neuro, we use it in cancer --

DR. GOUGH: Well, if it's good enough for neurology, why isn't it good enough for everything else?

DR. MICHALEK: I was trying to emphasize a point, and I guess I've made my point, so.

DR. STOTO: There's more to it than that. There are hypotheses and there is evidence that these things might be associated with cancer and with neurological effects, but is there any evidence that degreasing chemicals are associated with heart disease? No.

DR. MICHALEK: See, that's another thing.

DR. STOTO: The only one   

DR. HARRISON: Is there any evidence that drinking wine is associated with heart disease?

DR. STOTO: Yes.

DR. HARRISON: How long ago was that figured out?

DR. STOTO: Recently. I understand --

DR. HARRISON: I happen to think it's the solvent action of the alcohol, myself.

DR. STOTO: Well, it's always possible that there's some mechanism out there, or association that we don't understand. But generally just throwing in possible covariates just because there might be something there is not a good thing to do.

For them to be helpful, they have to have an association with the outcome.

DR. MICHALEK: Just to be some biological plausibility.

DR. GOUGH: Ten years ago there was no biological plausibility between dioxin and diabetes.

DR. HARRISON: I was just thinking a little earlier here today, I thought you were raising your eyebrow and talking about statistical stuff, and not really liking my argument for biology -- when I was talking about dioxin levels and everything you were saying "Who cares about the biological, it's the statistics that count."

(Laughter)

DR. FAVATA: Joel, I realize the limitations that you're discussing regarding the particular interviewer and their lack of ability to perhaps explain it, or the subject's ability to understand it. But I agree with Mike, also, if we're using it in neurology and feel confident in that; or should we even use it in that, if we don't have confidence in the information that we're getting.

DR. MICHALEK: I think we should come down on the side of using it.

DR. CAMACHO: Is it possible to have a short list like a prompt list in the questionnaire, like an example list?

DR. FAVATA: Would that be helpful to the interviewer?

DR. CAMACHO: Yes.

DR. HARRISON: It probably would be helpful for a lot of the questions, because it would standardize what kind of elaborations you could make.

DR. CAMACHO: Now that you said standardized, are all the interviewers given training?

DR. MICHALEK: Yes.

DR. HARRISON: Yes.

DR. FAVATA: And what is that training? What is the training.

DR. CAMACHO: Maybe that's an absurd idea, a prompt list as an example.

DR. MICHALEK: They are given prompt lists, as I recall, in certain areas but not others. In the area of contraception, for example: Have you ever used any contraception? And they'll give a list of 'such as' and there will be a list of several kinds.

MS. YEAGER: I think there is one for heavy metals.

DR. HARRISON: There is? Okay, then there may be lists for other chemical exposures, too.

DR. GRUBBS: For the record, these interviewers are not supposed to elaborate beyond their training.

DR. HARRISON: We understand that.

DR. GRUBBS: The standard approach.

DR. HARRISON: We understand that.

DR. GRUBBS: It has nothing to do with the skill of the interviewer, at all.

DR. HARRISON: But if there were a prompt list, then again   

DR. GRUBBS: It's part of the protocol, it's part of the procedure.

DR. MICHALEK: We have such prompt lists; I just don't have them at my fingertips at this point.

DR. HARRISON: Okay. Yes?

DR. FAVATA: Regarding the residential, how old is that EPA data?

DR. MICHALEK: I would guess about ten years, but I'll have to check.

DR. HARRISON: The point that I thought was more telling is that it's cobbled together; it is not done    it's not like the EPA did a soil sample analysis in every zip code or anything like that.

DR. FAVATA: It's cumulative over a battery of times.

DR. MICHALEK: I believe you'll find it comes from various studies, all kinds of sources. And it's modeled. There's a modeling.

DR. HARRISON: You know, one thing is that in biology there's an awful lot of bad data. I mean, that just    that's the essence of biology. As long as you acknowledge that it's got a big variation.

DR. STOTO: I'm not objecting to the quality of these data; I'm trying to think through the logic of including variables of this sort. There were a number of different kinds we've been talking about here in the analysis.

And I think that if we got halfway decent data that would give some indication about industrial exposure or other occupational exposure to a chemical that was suspected of being associated with the outcome, then we should use it.

By the my concern is that other than for cancer or neurological things, I don't know    maybe there's some evidence I don't know about, that these kinds of things are associated with the outcomes.

DR. FAVATA: Well, if you look at liver disease, you could have because of solvent exposure.

DR. STOTO: Okay. So that's the kind of thinking that   

DR. HARRISON: Especially a solvent plus an alcohol.

DR. STOTO: Okay.

DR. FAVATA: I think that there are a lot of health endpoints that they would affect.

DR. MICHALEK: Another area that we've explored in journal articles are exposures to various prescription drugs, and what they might have to do with the endpoint. In our latest neurology paper, we exclude individuals that have had certain chemotherapies, for example, because it was known by our collaborator that taking such medication is associated with a peripheral neuropathy in some people. So we need to do an exclusion.

So that whole area has not been fully explored in the study. The use of prescription medication --

DR. HARRISON: But see there are two separate things here. We're talking about, again, inclusion of data in the longitudinal analysis in the blue book.

You can go do an article on neuropathy and you can futz around and throw people out, and as long as you can convince the reviewers, that's your little red wagon, that's not a problem.

The question is, should the regional dioxin exposure database be used to see if it clarifies any relationships within the areas that are being studied? And it kind of sounds almost like it should, even though it's an imperfect database.

DR. STOTO: I think that even if the data were good, I don't think that the logic makes sense there, given that we have individual level measures of dioxin.

DR. HARRISON: But if you're looking at area under the curve, if you have area under the curve and you've got a zero dioxin environment and you've got a first degree curve--

DR. STOTO: But we're not talking about area under the curve for needs analysis here in the blue book.

DR. HARRISON: Well, I keep hear Joel keep talking about area under the curve. I'm not sure how he intends to include it; but the point is, obviously, that    if he raised the general environment level, then you're going to affect the area under the curve.

DR. STOTO: I agree that in a sophisticated analysis, that you could take it into account that it could be helpful. But in the models that we're talking about for the blue book, it doesn't seem to me to fit.

DR. HARRISON: "If it don't fit, must have quit."

DR. STOTO: Well, from the point of view of a skeptic might say "Well, you're missing an effect" because it could be that some of your group was living in a very heavily polluted area and another part of your group wasn't, and so that may have biased your results towards finding nothing because you didn't account for that disparity in environmental exposures.

DR. MICHALEK: But if people were living in a heavy polluted area, they would have high blood dioxins.

DR. STOTO: but if people were living

DR. MICHALEK: Well, true.

DR. GOUGH: Actually, it's very hard to    in a general population exposure to generate a high dioxin level.

DR. STOTO: Then it doesn't matter.

DR. GOUGH: I know. That's what bothers me, is that it's --

DR. HARRISON: Let's ask the question another way. You don't think that it will be very helpful. Do you think it's wrong?

DR. FAVATA: What is "it" that we're talking about?

DR. HARRISON: Using the regional adjustment, using that as a, adding that as a covariate. Do you think it's wrong or do you think it's just not very   .

DR. STOTO: The problem is you have a Y variable which is a disease. You have an X variable which is either a dioxin or being in the Ranch Hand group versus the control    that's the primary exposure we're trying to study. Then you have some Z variables you're trying to adjust for like body mass index and so on.

DR. HARRISON: Or another Z --

DR. STOTO: Well, it's not clear whether we're throwing in an X or a Z. That's my problem, because it's a dioxin measure    if in fact the relationship is due to dioxin, if living in the area    if you lived in Times Beach and you got a lot of dioxin that way, and it caused diabetes, that showed that there is a relationship between    it adds to the strength of the relationship. You don't want to take that away, you want to capitalize on that.

So I guess that it would tend to take away from the relationship between diabetes and outcome.

DR. HARRISON: Well, maybe in the executive summary of the minutes of this meeting, it can be said that the committee was divided in its assessment of whether this would add value to the analysis of the 2002 data.

Is that fair enough? I want to move us off of this and onto something else, is what I'm trying to do.

DR. FAVATA: After hearing what Joel said as far as the EPA data, it really doesn't sound strong enough to go forward and use that.

DR. HARRISON: The committee is not divided. The committee doesn't think it's a good idea.

DR. GOUGH: But we considered it. I think it's worth noting.

DR. HARRISON: Yes. We considered the heck out of it.

DR. STOTO: I have become convinced, though, about the value of using the questionnaire about occupation and --

DR. GOUGH: Yes, I would like to see that.

DR. STOTO: Because that really is adding another Z, because it's   

VOICE: (inaudible)

DR. STOTO:    yes, and it's because it's exposure to other things that are primarily --

LTC BURNHAM: It reminded me of this when you mentioned that we have the dioxin levels. The plan is not to do it in '02.

DR. HARRISON: I've got that noted down here.

` LTC BURNHAM: So I don't know if you want to discuss that or not. But when you said "Well, we have a level so it really would wash with where they live, that may have an impact on the fact that we're not doing it again in '02.

DR. HARRISON: Yes, but most of the area data is older than your dioxin data.

` LTC BURNHAM: The other thing that I just remember is, I don't know what your plan is, but when you do give us suggestions    I don't know if we need to iron out the specifics here; maybe that would be best, or if you want us to get with an individual person after the meeting. But again we're trying to iron out this statement of work, and so it's tough for us to say to SAIC, "And oh, by the way, you need to throw in some stuff about where they live."

You need, we need to really word down to the specifics right now; we need to tell them exactly what we want.

DR. HARRISON: Well, and I think you all need to help us out in the sense of, if we think that we've made a recommendation and it's not clear, it doesn't contain enough    it's not granular enough, then you've got to pull our chain before we get away.

DR. STOTO: And I think that in some cases, it may make sense --

DR. HARRISON: To go individual.

DR. STOTO: To go individual.

DR. HARRISON: What I'll try to do is, if it looks like that would be the best way to work, I'll try to move it there so that we can keep on with our agenda, which is falling further and further behind.

I had noticed, though, I'm glad you mentioned that. I had down here serum dioxin draw, page 12: Will there be none this cycle? If so, why not delete it? I mean, you've got all this stuff in there about dioxin, but if I read it right, you said there are going to be zero samples.

DR. MICHALEK: No, no. First of all, 99 percent of the study subjects have at least one dioxin measurement already. We just meant that we weren't going to do    the only people that will get a dioxin level measurement this time are those that don't already have one.

DR. HARRISON: The contractor shall draw serum following the protocol    on up to zero participants.

DR. MICHALEK: That's a misprint, should be up to 40 participants.

DR. STOTO: That makes a big difference.

DR. GOUGH: Good catch.

DR. MICHALEK: Sorry. What page was that?


DR. HARRISON: Well, now that you're looking at it, it may be that the crossbar in the 4 and the crossbar in the X-out    it's 3.2.1.8
1   2   3   4   5   6   7   8   9   ...   23

Похожие:

Ranch hand advisory committee iconAdvisory committee on immunization practices

Ranch hand advisory committee iconMedical Devices Advisory Committee

Ranch hand advisory committee iconVeterinary medicine advisory committee

Ranch hand advisory committee iconNational Vaccine Advisory Committee (nvac)

Ranch hand advisory committee iconExternal Advisory Committee on Cities and Communities

Ranch hand advisory committee iconWildlife Diversity Policy Advisory Committee

Ranch hand advisory committee iconPeer reviewed by the Arizona Department of Commerce Economic Research Advisory Committee

Ranch hand advisory committee iconFood and drug administration national institutes of health advisory Committee on: transmissible spongiform

Ranch hand advisory committee iconAdvisory Committee, Cuyahoga Valley School-to-Career Consortium, Broadview Heights, Ohio 1996-2002

Ranch hand advisory committee iconJane D. Siegel, md; Emily Rhinehart, rn mph cic; Marguerite Jackson, PhD; Linda Chiarello, rn ms; the Healthcare Infection Control Practices Advisory Committee

Разместите кнопку на своём сайте:
Библиотека


База данных защищена авторским правом ©lib.znate.ru 2014
обратиться к администрации
Библиотека
Главная страница