Tech Talk – Medical Device Particle Test Method Validation

See the previous Tech Talk for Medical Device Particles.

As part of their focus on particles, the FDA has required test method validation of your particulate testing.  Their method validation is described as follows: 

“You should describe and validate particle counting and sizing methods. We recommend that you introduce a known amount of various particle sizes into the test setup and quantify the amount of particles recovered. The number of particles recovered should closely approximate the number you artificially introduced into the system. For a system to be considered validated, ≥90% recovery should be demonstrated for the ≥10 μm and ≥25 μm size ranges.”

Why do they say this?  In my experience, larger particles settle to the bottom of your sample container.  So if you don’t do a proper validation and sort it all out you will not get an accurate count of the most important particle sizes.  In older tests from various companies I have reviewed this was actually happening.

So where to start with this?  First off, read ASTM F2743 for a general guideline.  You will want to do this testing in house, the test labs (NAMSA, WuXi AppTec, Nelson, etc.) will all do it but it will be expensive, most likely you will have to travel to their site, and you have to validate the model you are using your device with anyway.   So just buy a HIAC 9703, which is the same machine the test labs use, you can call them and confirm- maybe they have upgraded by now. 

You also want a good supply of low particulate water for your testing.  Low particulate water can be just reagent grade water, buy from any chemical supply company for cheap.  You also may have an internal system that can produce low particulate water, just check in your HIAC, although it is probably less effort to just buy the bottles of water.  All testing / flushing and rinsing should be done using this water.

During the testing you will probably want some clean glassware, it is actually not that hard to keep the glassware clean during the testing, rinse a clean container with low particulate water a few times and you’re probably good to go.  Just don’t dry anything with paper towels; you’re better off air or shaking it dry.  The USP standard requires a laminar flow hood, but you’ll probably be okay as long as the room is reasonable clean and you shut down the wood sander before you start.

Then you’ll want to buy some particulate standards so you can do your validation, the only source I’ve found for these is Fisher Scientific, I prefer the Count-Cal particles and will assume you use these particles.  This is where you may want to think about it some or just buy the following sizes 10 um, 15 um, 30 um, 70 um and 100 um.  You can skip the 10 and 100 sizes and still meet the USP and FDA guidance, but if you’re going to validate the method, you might as well do it only once and not worry about it again.  If you read the literature on the particle standards you will see that for example the 15 um size has all particles above 10 um.  So the 15 um size will validate the 10+ um bin, etc.  The 10 um size is nice to have to show that the under 10 bin is working properly (i.e. you are not counting everything).  I’ve seen people assume the 10 um standard is a normal distribution and that the count above 10 um is one half of the total count, I think this is a questionable assumption and it is better to use the 15 um size to validate the 10 um bin.

You will also note by reading the literature that comes with the standards the count is not calibrated, only the particle size.  So you cannot accurately use a particle count value calculated from the standard literature to compare against, you should measure the standard yourself and use that as your baseline.  This is important to understand and you’ll probably have to explain it to many people in management and quality who cannot be bothered to think about it beforehand.

You’ll want to create a custom test using the HIAC software, measure the particle sizes you’re planning on calibrating to (i.e. at least 10, 25, and 50, probably 70 and 100), discard the first run and display total count and run count. 

The literature that comes with the particles standards says to discard the first run on the small particle sizes, which the machine can do automatically.  The literature that comes with the particle standard also says to discard the last run on the large particle sizes, but this assumes you are collecting all liquid in your sample- you cannot do this if you’re mixing with a stir bar.  You can discard the first and last runs manually, or discard just the first run and make sure you leave some sample in the sample container when the runs are completed.

I would set your test up to do 5 runs total of 5 ml each and discard the first and last run you will have to discard the last run manually.  5 runs require 25 ml of sample plus enough extra to clear the stir bar by a bit.  I’m basing my experience off of small devices, if you’re testing another type of device, adjust the volumes as appropriate, but I would not use a smaller volume.

For the test method validation, your tests should go something like this:

        1.   Wash methods, glassware and HIAC

        2.   Water through model / tortuous path (results will be used as baseline)

   3.   Particle standard (start with largest)

        4.   Remaining particle standards

The first test will be low particulate water in the sample container (ideally a 100 ml beaker- see below).  You want to ensure your water is okay, your cleaning method is good, and your HIAC is clear.  The test you just made should be used for all testing here on out, the standard tests don’t include all particle sizes and are of limited utility.  I would perform these steps to test:

        1.  Put beaker on HIAC stand, align with tape so it is consistent every time

        2.  Put clean stir bar in beaker, cover with parafilm, stir at highest speed setting

        3.  Wait for two minutes while stirring (recommended in USP 788)

        4.  Run test

If you get poor results, then you generally want to rinse more, using soap to wash during particulate testing doesn’t always help unless you’re desperate and willing to rinse a lot.  Using IPA can help but also requires a lot of rinsing, IPA should not be used except when needed, do not use it between every test.  Starting with clean glassware and rinsing a lot with the reagent grade water is your best bet.

Let us talk about mixing the samples briefly.  Gently shaking or inverting the samples is not adequate.  Without stirring with the stir bar while the HIAC is testing, larger particles (50+ um) will settle.  You can easily test this using a large particulate standard.  If you test while stirring you will get a higher particle count overall as well, is this from the stirring or did they settle previously and now you’re counting them?  It doesn’t matter as long as your baseline is consistent with your test.

Once you’ve done this, you’ll need to do some thinking.  How are you going to run your particulate tests long term?   Basically this breaks down to what sample volume you can collect, which depends on your model and device size.  I think ideally you want to end up with at least 50 ml of sample.  One half of the sample will be used to flush the model after use, so you have 25 ml of water to work with for the testing.  A 50 ml sample fits nicely in a 100 ml beaker with a stir bar and the HIAC sample collection probe fits in while still allowing stirring.  You’ll want to make sure the probe is not too close to the stir bar as the stir bar does generate particles (or bubbles which are counted as particles).

A typical set up for a catheter would be your appropriate tortuosity model, with fittings on both ends, on the proximal end a touhy borst with a Y fitting, through the touhy borst is a standard guide catheter.  Your device is delivered through the guide catheter, additional accessories devices may be used if used with your product.

Your typical use might end up like this, fill model with 10 ml water, fill guide catheter with 1 ml water, perform simulated use with your device which takes 4 ml of water (obviously varies by device volume).  Flush guide catheter with 10 ml of water, flush model with 25 ml of water.  There is no magic to the quantities, you just want a complete flush of the system you’re testing and to get around 50 ml or more of sample water.  Going forward, I’m just going to assume your water volume is 50 ml.   You may also flush through your device if appropriate, you should be careful extra steps have a tendency to generate particles and if they’re not done clinically, you’re just asking for trouble. 

An example of a simulated use for a catheter would be to prepare your catheter (i.e. remove from packaging and hydrate), place your catheter in the guide catheter, advance and retract over a guide wire through tortuosity several times and maybe deliver a stent.  In this case you would definitely flush through the model as the distal end of the catheter is exposed to circulating blood.  If the clinical use was to flush through the guide catheter, you would flush through the guide catheter as well.  If the clinical use was to flush through the catheter (i.e. a contrast injection), you would flush through the catheter as well.

Before testing your device, verify that your model is not generating significant amounts of particulates and use it as a baseline.  You’ll want to make sure it has no dead zones before starting and it is best if the model can be easily drained.   If you’re using a guide catheter or other accessory device, you may want to include that in the baseline, I would do this.  When choosing what accessory devices to use in your baseline, use clinically relevant devices, but also chose ones that are unlikely to generate particulates.  You want a solid guide catheter and guide wire, not something that generates a large number of particles that can obscure the results from your device.  If the accessory devices generate too many particles you may consider other alternatives, like plain PTFE tubing if that is reasonable. 

Once your model is set up with associated fittings and any accessory devices you want to include in the baseline, inject 50 ml of water through, collect the effluent in the clean sample container and drain the model into the sample container.  Perform your test on the sample, I would repeat three times, average each bin, and use this as your pre-test baseline.  You’ll probably want a post-test baseline as well, or alternatively you could take a pre-test baseline prior to every test.  Your model should really have very few particles and hardly ever one 50+ um.

Some people want to collect all their samples, and then test them all, but I feel pretty strongly that you should test your samples as you obtain them.  Letting them settle in the sample container isn’t going to do you any favors down the road, you’ll probably get low particle counts now, but when you need to do some comparison testing or need to make a change it will be more difficult to reproduce. 

To validate your test and model, take the 70 um particle standard bottle and make a “standard solution”, to make the standard solution:

         1.   Shake the 25 ml bottle of 70 um standard solution vigorously for 10+ seconds

         2.   Pour entire 25 ml of 70 um standard solution into a 400 ml beaker

         3.   Pour 200 ml of LPW into beaker

         4.   Pour 25 ml of LPW into the empty standard solution bottle, rinse and then pour into the beaker

         5.   Put clean stir bar into 400 ml beaker and stir at a moderate speed, do not stop stirring

         6.   Cover with parafilm when not in use

Now you’ll want to test 50 ml of your 70 um standard solution using the procedure above and see what you get at the readout from the 50 um size (the 70 um particulate standard is 100% above 50 um), you should be ballpark of the bottle count at the 50 um size (after you take into consideration the dilution we did) and all of your runs 2-4 should be consistent, use the average of the runs.  You should be about half of the bottle count at the 70 size, but that is less accurate and I wouldn’t sweat it too much.  If you see your last run spike then you’re probably too close to the stir bar and you may want to consider increasing your sample size or raising the HIAC sample intake if possible, if you do this, repeat the test. 

Once you’re happy with those results you can inject 50 ml of the standard solution into your model, collect the effluent and see how you do.  Average the runs from your test (discarding which runs you said you would).  Subtract your baseline result average before you calculate the amount recovered.  You want to recover more than 75% of what you put in per the FDA guideline.  The 70 um particle size is the most challenging, which is why we started here, so don’t worry too much if you don’t get it the first time.  If you’re not recovering at least 75% of your starting particles- your particles most likely have settled in your standard solution.  Turn up your mixing on the standard solution and start over.  You’ll want to proceduralize the mixing.  If you recover more than 110% you probably want to look at your environment and wear a hair net or breathe more through your nose or something.

Repeat with the rest of the particle standards, you want more than 90% recovery with at least the 15 and 30 um sizes and you have finished the particle testing test method validation.

See Part 3: Particulate Testing of Medical Devices.

Tech Talk – In Vitro Medical Device Verification Testing in Blood

I previously discussed testing medical devices in blood here (in 2007!), but I think I did a poor job of it and I’d like to revisit it.

Why do you test in blood? Well for one, blood is hard to simulate, it’s a non-Newtonian fluid, and using glycerin and water don’t really do it justice, but these can work depending on the application. For another, a common blood test is to check for hemolysis, sure this is tested during a biocompatibility test, but biocompatibility tests are not performed during actual use conditions. Hemolysis may also be a part of an animal safety study that you want to check out beforehand.

Where do you get the blood from? At a slaughterhouse of course, if you can find a smaller or craft meat location in your area, they’ll probably work with you, one used to sell to us for $40 a week, and we’d take a couple gallon buckets and their workers would fill them up while we waited. They only slaughtered on certain days, so call ahead. You’ll probably find cows easier to find and work with, but there isn’t really a reason you couldn’t use pig blood.

Before we leave for the slaughterhouse we’ll set up a water bath at 37ºC to be ready when we get back. Then we’ll add anticoagulant to the blood collection bucket. We’ll use either heparin or Acid Citrate Dextrose (ACD).

Once we get the blood, we mix the bucket to ensure the anticoagulant is distributed in the blood. Heparin is prescription drug, so hit up your vet consultant or animal lab for some ahead of time. ACD you can make based on USP guidelines from commonly available chemicals (water, citric acid, dextrose, and sodium). We used around 10,000 to 20,000 units of heparin per liter of blood. Of note is heparin is used clinically (on people) more in the U.S. and ACS is used in Europe, so you could maybe argue for the use of one over the other, but you’re using animal blood, so I’m not sure if that really matters. I’ll assume we are using bovine blood for the rest of this post. If you don’t use an anticoagulant, you’ll end up with a clot bucket when you get back to the lab, just throw it away if this happens, it is not recoverable.

Time is generally of the essence so don’t stop by Chili’s on your way back to the lab. Also, just be aware that water will damage your blood cells, so it is preferable to rinse your lab ware with a bit of saline before use.

When we get back to the lab we first check the blood pH and temperature, ideally the pH is between 7.2 and 7.4. We then take a hematocrit (hct) measurement by collecting blood in a capillary tube with clay sealant to stopper the bottom (get blood before using clay). Then we centrifuge the capillary tube for a few minutes at high rpm. Once centrifuged, the capillary tube will look like this:

You’ll need a hematocrit chart. Below is a simple representation of how to measure hematocrit, you put the capillary tube on the chart, line up the clay on the baseline, you move the tube left or right until the fluid level matches the top line, then you find the line where the red blood cells stop and follow it over to read the percent hematocrit, in this case 50%.

We generally take two hematocrit measurements and average; you need two capillary tubes to balance the centrifuge anyway. 38 to 42% hct is a typical range for a study like this one, although it will vary depending on how much the animal drank before it was slaughtered, I’ve seen it come in in the low 20s, so don’t worry about the initial hematocrit too much.

We then pump the blood from the collection bucket through a saline primed arterial filter (pediatric filters have lower priming volumes) and line to remove hair and large clots and into a carboy with a plugged outlet at the bottom. We’ll set up a circuit from the bottom of the carboy to the top (through the filter) with a peristaltic pump to keep the blood circulating. Place the outlet in the blood and not above it or you’ll get a bunch of foam. At this point we’ll add saline to the blood to get the hematocrit where we want it, usually around 22% to 32%. Keeping hematocrit consistent is better than not. We’ll measure the hematocrit and adjust until we’re good, using the following formula:

 S = [(H/F)-1]xV

 Where:

 H is the initial hct,
 F is the desired hct,
 V is the original volume of blood, and
 S is the volume of saline to be added.

Once we get the hct where we want, we’ll measure pH and temperature again. At one point we were centrifuging the entire sample to remove the buffy coat layer between the serum and the cells, then mixing it back together but this proved pointless and didn’t really benefit our results or affect our testing any and it was a major pain, so I don’t recommend it.

Once prepared, we can expect the blood to last for five or so hours before it gets questionable. If we’re testing an endovascular device, we’ll pump the blood around a tubing circuit (using a peristaltic pump) and then place the device in the tubing. Preferably the tubing is a similar inner diameter to the artery or vein the device will be used in. You probably want to place the blood reservoir above the test set up and the pump after the test area. Putting the blood reservoir above the test set up ensures a more consistent blood flow. A simple set up is shown below.

We can measure the device performance in blood directly, or we may be interested in something like how much does the device damage the blood, we’ll check the serum and see how red it is in simple terms. If it gets worse over time, then we’re damaging the blood. In this case, for an accurate comparison we need to run a control at the same time. For example, if we have an elaborate pumping system, we’ll run our pump system on one closed circuit and the control (with no device) in another closed circuit and track the hemolysis of both over time.

Bench Testing

I've spent the last few weeks dealing with bench tests as we wait for other results to come in. Bench testing or performance testing gets stuck in section 18 of the 510(k), since it is in the back means they don't read it right? You will save yourself a lot of grief if you read the format guidance and make sure your protocols and reports line up nicely with their requirements, although you will need more than what is listed in the guidance, at least a scope.

For a small company there are several challenges to bench testing, all revolving around the number of people that have enough understanding to run the tests. My company has three plus one consultant that can run the majority of the tests, it is preferable to have employees sign off on everything so the consultant is out, and one of the three has the understanding to run the tests, but not the personality type to see it through. That leaves the two engineers, one of which is on vacation this week, so that leaves me for now. MD&DI sums up the who should do the bench testing very well.

The first problem is the protocol, which must be signed off before the test begins, the problem here is that no one besides the engineer authors are likely to really understand what is going on. This means no problems will be caught until the engineer testers try it for real. Sure, we've tested it some previously, but when everything is recorded things change. I wrote a protocol and discovered I couldn't hold a negative pressure I thought I could so had to change it up a bit. This means rewriting and walking around getting signatures to get it approved before I can start again. This is not much of a problem, unless it is after 3:30pm and QA has gone home for the day. Then I'm forced to wait around until they come in at 9 the next day. I have argued that by having my signature on it that the protocol has therefore been predefined and good to go, but I haven't gained much ground with that.

The next problem is that these tests take time, we are shooting for 24 hours of use. I rallied around testing for 26 hours but my boss vetoed that saying 1.5 times is standard, meaning 36 hour tests and every other day I have to come in at an awkward time (do not worry, I am getting my revenge- see below). I am amazed my wife hasn't accused me of cheating on her yet with the late night stops by work. The 1.5 times the maximum limit you're shooting for is a good rule of thumb, and appropriate here, but it doesn't work for everything, like negative pressures.

The last of my whining centers around the sample sizes that will not be high enough to make everyone happy. With limited product and limited resources, running a dozen 36 hour tests could take a month. Unless you are going to manufacture, sterilize, and shipping simulate a batch of samples yourself in the next week, complaining about sample size doesn't accomplish much. Do a reasonable job and if the FDA picks on it the most likely thing that will happen is they'll ask for more testing.

I mentioned in my previous post that the deadline slipped (still not my fault), this has given me time to come up with some additional bench testing to put in motion. I say put in motion because I was so confident I'd meet my part of the original 510k deadline that I planned a two week Hawaii trip starting one day before the deadline. Now all the loose ends will have to be tied up by my boss and the other engineer, I sorta feel guilty now, but about 20 minutes after landing it will be forgotten. I give the extra testing a 40% chance of not being done when I get back. I have to say though that the last year has been a blast and if you're an engineer with a good work ethic that can tolerate the risk of working for a smaller company then go for it.

In-vitro tests

Here at the happy medical device factory we've moved on from sterilization to our next crisis, the in-vitro test. There are still quite a few outstanding sterilization issues, but we like to make things interesting by panicking about something else before the last crisis completely subsides. The in-vitro test's goal is to obtain data that supports the safety and efficacy of the medical device.

Usually when a test is described as in-vitro it involves using blood outside of a body. Keep in mind when I describe it, I'm an engineer, so as usual- I probably don't know what I'm talking about. In our case, we run down to the local slaughterhouse and pick up gallons of cow blood, add an anticoagulant, heparin for us, (you can also use sodium citrate) and bring it back to use for our test.

When we get the blood in we run it through an arterial filter, add dextrose (to preserve- that is the theory anyway), check the temperature, pH and hematocrit. The pH we want between 7.2 and 7.4, but it depends on temperature somewhat, we're currently in disagreement about what we should do if the blood comes in above 7.6 or so, I'm in the throw it out and try again tomorrow camp, but others have argued for adjusting it. Luckily, we haven't had any pH above 7.6 in years. The hematocrit (hct) we generally get 38-42% and we adjust it down to 32% by adding saline, a small vain attempt to keep at least one thing consistent across tests. Another common tactic is to adjust hct to 25%. A previous incarnation of this test had us removing the buffy coat layer of the blood, but that required a lot of time and effort that did not really improve the results.

The next step is the moment of truth for the whole day, splitting the blood into the sample groups, for us, our device, predicate device, and control. I say it is the moment of truth because once you split it into the three containers or whatever you're testing in, you test the blood variable you're looking at in the test. You hope and pray that these initial readings of whatever blood variable you're testing come out reasonably close together, if not you just wasted half a day. It sounds easy, but blood is not consistent and any number of small factors can mess things up, blood settling, not perfectly clean containers, water in the sample containers, etc.

Once we have that down the test can begin in earnest for the next 5 or so hours. The reasonable life of blood is generally 6 hours. After that is clean up, it makes for a long day. Entertainingly enough now, on a previous 72 hour device I worked on the FDA required us to do an in-vitro test for 72 hours, saline tests weren't good enough for them, the results were predictable, a black, stinky soup that stunk up the place for weeks after it was disposed of, but hey, we got our largely irrelevant results!

My main task after an in-vitro test seems to be discouraging people from comparing results across in-vitro tests, but that is a post for another day.